Install Redash server On-Prem on Debian

I followed setup instructions to install the redash server on-prem. There’s a provisioning script to install the redash server on-prem. But as expected, things don’t really work first time on all environments. I used the latest Debian (stretch) v9.2.1 and ran into issues running the provisioning script. Then I had to manually debug and go through each step/package installation at a time and found the following 2 issues.

1. pip installation cryptography didn’t work. I eliminated this step. Remove the following line from the file /opt/redash/current/requirements.txt


2. The pyOpenSSL package wasn’t compatible so I had to replace the version in requirements_all_ds.txt file under /opt/redash/current/ directory


Original Script for reference

# This script setups Redash along with supervisor, nginx, PostgreSQL and Redis. It was written to be used on 
# Ubuntu 16.04. Technically it can work with other Ubuntu versions, but you might get non compatible versions
# of PostgreSQL, Redis and maybe some other dependencies.
# This script is not idempotent and if it stops in the middle, you can't just run it again. You should either 
# understand what parts of it to exclude or just start over on a new VM (assuming you're using a VM).

set -eu

REDASH_BRANCH="${REDASH_BRANCH:-master}" # Default branch/version to master if not specified in REDASH_BRANCH env var
REDASH_VERSION=${REDASH_VERSION-2.0.1.b3080} # Install latest version if not specified in REDASH_VERSION env var

cd /tmp/

verify_root() {
    # Verify running as root:
    if [ "$(id -u)" != "0" ]; then
        if [ $# -ne 0 ]; then
            echo "Failed running with sudo. Exiting." 1>&2
            exit 1
        echo "This script must be run as root. Trying to run with sudo."
        sudo bash "$0" --with-sudo
        exit 0

create_redash_user() {
    adduser --system --no-create-home --disabled-login --gecos "" redash

install_system_packages() {
    apt-get -y update
    # Base packages
    apt install -y python-pip python-dev nginx curl build-essential pwgen
    # Data sources dependencies:
    apt install -y libffi-dev libssl-dev libmysqlclient-dev libpq-dev freetds-dev libsasl2-dev
    # SAML dependency
    apt install -y xmlsec1
    # Storage servers
    apt install -y postgresql redis-server
    apt install -y supervisor

create_directories() {
    mkdir -p $REDASH_BASE_PATH
    chown redash $REDASH_BASE_PATH
    # Default config file
    if [ ! -f "$REDASH_BASE_PATH/.env" ]; then
        sudo -u redash wget "$FILES_BASE_URL/env" -O $REDASH_BASE_PATH/.env

    COOKIE_SECRET=$(pwgen -1s 32)

extract_redash_sources() {
    sudo -u redash wget "$LATEST_URL" -O "$REDASH_TARBALL"
    sudo -u redash mkdir "$VERSION_DIR"
    sudo -u redash tar -C "$VERSION_DIR" -xvf "$REDASH_TARBALL"
    ln -nfs "$VERSION_DIR" $REDASH_BASE_PATH/current
    ln -nfs $REDASH_BASE_PATH/.env $REDASH_BASE_PATH/current/.env

install_python_packages() {
    pip install --upgrade pip
    # TODO: venv?
    pip install setproctitle # setproctitle is used by Celery for "pretty" process titles
    pip install -r $REDASH_BASE_PATH/current/requirements.txt
    pip install -r $REDASH_BASE_PATH/current/requirements_all_ds.txt

create_database() {
    # Create user and database
    sudo -u postgres createuser redash --no-superuser --no-createdb --no-createrole
    sudo -u postgres createdb redash --owner=redash

    cd $REDASH_BASE_PATH/current
    sudo -u redash bin/run ./ database create_tables

setup_supervisor() {
    wget -O /etc/supervisor/conf.d/redash.conf "$FILES_BASE_URL/supervisord.conf"
    service supervisor restart

setup_nginx() {
    rm /etc/nginx/sites-enabled/default
    wget -O /etc/nginx/sites-available/redash "$FILES_BASE_URL/nginx_redash_site"
    ln -nfs /etc/nginx/sites-available/redash /etc/nginx/sites-enabled/redash
    service nginx restart


3. Setup mail server
When you open redash, you’ll get a message notifying that the mail server has not been setup. The setup page doesn’t have much details about how to setup mail configuration and troubleshoot. After some trial and error, I was able to configure the mail server properly.

Test sending email from console. This would throw an error if the configuration/setup is incomplete and notify what’s missing.

cd /opt/redash/current
bin/run ./ send_test_mail

The Setup Page has details about some environment variables that need to be setup for Mail Configuration. But just exporting them on the console didn’t work. There are a couple of places that you can enter these settings – a .env file that stores the environment variables (PREFERRED) and a settings file that reads from the env file. Prefer storing the configuration in the env file so settings stay pristine and is not hard-coded. Here is a detailed list of Environment Variables with descriptions that redash supports and can be configured.

# Setting redash environment variables (PREFERRED)

# Settings file that reads from the Environment Variables.

After setting the Environment Variables, do have to restart all processes (not just nginx)

# Restart all processes
sudo supervisorctl restart all

# Restart the Web server
sudo supervisorctl restart redash_server

# Restart Celery workers
sudo supervisorctl restart redash_celery

This maintenance page has information about ongoing maintenance and troubleshooting steps for the redash server.


Installing / Re-Installing Apache, PHP, libapache2-mod-php

The following is done on an Ubuntu machine but hoping it should cover other Debian instances.

I accidentally deleted php7.0.conf and php7.0.load from the mods-available folder. These files come with libapache2-mod-php. I uninstalled the module and installed it again – but the files didn’t appear. A lot of blogs suggested to remove Apache, PHP and libapache2-mod-php altogether and installing it again and it didn’t work. I used remove, purge and everything suggested but didn’t help. Eventually, I had to search for the installed module by:

dpkg --get-selections | grep libapache2-mod-php

I installed the module above as is but the result showed me libapache2-mod-php7.0. So I went ahead and removed and purged the module listed after which I installed the module again (without the 7.0 suffix) and I was able to recover the files again.

apt-get remove --purge libapache2-mod-php7.0
apt-get install libapache2-mod-php

Since I am covering this topic, let’s talk about installing Apache on Debian and the supported PHP packages along with SSL support.

apt-get install apache2
apt-get install php
apt-get install php-curl
apt-get install libapache2-mod-php

Enable SSL on the site:

cd /etc/apache2/sites-enabled
ln -fs ../sites-available/default-ssl.conf 

Finally, add the necessary modules to support PHP/SSL on Apache

cd /etc/apache2/mods-enabled
ln -fs ../mods-available/ssl.conf
ln -fs ../mods-available/ssl.load 
ln -fs ../mods-available/socache_shmcb.load
ln -fs ../mods-available/php7.0.conf
ln -fs ../mods-available/php7.0.load

Do not forget to restart apache with

service apache2 restart

Linux – Adding useful aliases

I prefer to add the aliases to a different file rather than the .bashrc. You can modify the .bashrc file to include the .bash_aliases file:

# Alias definitions.
# You may want to put all your additions into a separate file like
# ~/.bash_aliases, instead of adding them here directly.
# See /usr/share/doc/bash-doc/examples in the bash-doc package.

if [ -f ~/.bash_aliases ]; then
    . ~/.bash_aliases

And then create a new file .bash_aliases in the home directory with the following or your own aliases:

# Adding custom aliases
alias l='ls -lrt'
alias lwc='ls -lrt | wc -l'
alias la='ls -lart'
alias lm='ls -lrt | more'
alias lwc='ls | wc -l'
alias f='find . -name $1'
alias b='cd ..'

Finally, to execute the bash_aliases in your current session without having to logout and login again, execute the following command:

. ~/.bash_aliases

Rename files in a sequential order

My camera shoots images until DSC_9999 and after that it resets to DSC_0001. It messes up the sequence of images shot during a day if the sequence is reset. I found this cool solution which resets them in sequential order starting from 1 and also pads 0’s to the beginning if the number is less than 100.

for i in *.JPG; do
  new=$(printf "DSC_%04d.JPG" "$a") #04 pad to length of 4
  mv -i -- "$i" "$new"
  let a=a+1

Debian – Switching the hostname/IP of the machine cleaned up the resolv.conf file

I built a new server to replace an existing staging server. After I did the cutover and made the new one my staging server, I realized I wasn’t able to ping internally on the network as well as outside. I checked the networking on the VMware server and it was set correctly but I still kept getting the following error on ping:

$ ping
ping: unknown host

I checked the hosts file under /etc/hosts and the hostname file and both of them were fine. However, I tried to ping the IP of google and I was able to connect. Which made me realize that resolving of the domain names to IP via DNS (Domain Name System) is not working.

Next, I did a nslookup for an internal system and google and didn’t get any response:

$ nslookup
;; connection timed out; no servers could be reached

And then I tried doing a nslookup by using Google’s Name Server A Name Server is a specialized server on the Internet that handles queries or questions from your local computer, about the location of a domain name’s various services.

A great simple way to think about name servers is using a phone book analogy. If you were trying to call Joe’s Trading Company, you’d want to look it up in a phone directory to find the phone number. The Name Server or DNS Server as it is also called does the same thing with finding out IP addresses for websites/DNS entries.

But anyways, here’s the output of nslookup when I used Google’s Name Server:

$  nslookup

Non-authoritative answer:

So, I came to a conclusion that the DNS resolution wasn’t working. I checked out the resolv.conf file on my local machine and realized it had been blanked out – possibly during the host/IP switchover. I checked another working server and copied over the configuration to my local file under /etc/resolv.conf. resolv.conf is the name of a computer file used in various operating systems to configure the system’s Domain Name System (DNS) resolver.

domain company.pvt
search company.pvt
nameserver 192.168.XXX.XX
nameserver 192.168.XXX.XX

After saving the file, I did a nslookup and got the following result:

$ nslookup
Server:		192.168.XXX.XX
Address:	192.168.XXX.XX#53

Non-authoritative answer:

A little bit more about DNS requests and how they work
But just how exactly did your computer know what webpage to display for you, and what server to pull it from?

Your web-browser knows you typed into the address bar.

Your current nameservers from the resolv.conf file are retrieved.

Your computer asks the nameservers for the A (address) record for

The nameservers respond back with the IP address

Your computer sends a request to that IP address along with the page you're requesting.

The web server hosted on then sends your web browser the requested page.

Note that on occasions, there might be canonical names (CNAME) assigned to a website. If you do a nslookup on the website, the nameserver will respond back stating that the requested entry is a canonical entry and then provide the address (A) record for the main server. e.g. Google uses WebEx’s services and have a unique URL for their operation. They may not have a standalone server but may be sharing the same WebEx’s server with other clients of WebEx. I did a nslookup on and the result confirmed that the dns was a canonical (CNAME) entry

$ nslookup
Server:		192.168.XXX.XX
Address:	192.168.XXX.XX#53

Non-authoritative answer:	canonical name =	canonical name =