Python – Export to CSV

Exporting data to a CSV file:

import csv

# Create CSV file
f = open('file_name.csv', 'w')
file_writer = csv.writer(f, quotechar='"', quoting=csv.QUOTE_MINIMAL)

# Write headers to CSV file
volume_header = ['col1', 'col2', 'col3', 'col4', 'col5', 'col6', 'col7']
file_writer.writerow(volume_header)

# Simplifying the code assuming you have an array of data
for i in array:
	file_writer.writerow([i.val1, i.val2, i.val3, i.val4, i.val5, i.val6, i.val7])
f.close()

Python smtplib – Send email with attachments

Here’s how you can send attachments from email using smtplib

#!/usr/lib/python
import smtplib
import mimetypes
from email.mime.multipart import MIMEMultipart
from email import encoders
from email.message import Message
from email.mime.audio import MIMEAudio
from email.mime.base import MIMEBase
from email.mime.image import MIMEImage
from email.mime.text import MIMEText

emailfrom = "FROM-EMAIL@DOMAIN.COM"
emailto = "TO-EMAIL@DOMAIN.COM"
fileToSend = "UPLOAD-FILE.csv"

msg = MIMEMultipart()
msg["From"] = emailfrom
msg["To"] = emailto
msg["Subject"] = "YOUR EMAIL SUBJECT"
msg.preamble = "YOUR EMAIL SUBJECT"

ctype, encoding = mimetypes.guess_type(fileToSend)
if ctype is None or encoding is not None:
    ctype = "application/octet-stream"

maintype, subtype = ctype.split("/", 1)

if maintype == "text":
    print 'text'
    fp = open(fileToSend)
    # Note: we should handle calculating the charset
    attachment = MIMEText(fp.read(), _subtype=subtype)
    fp.close()
elif maintype == "image":
    print 'image'
    fp = open(fileToSend, "rb")
    attachment = MIMEImage(fp.read(), _subtype=subtype)
    fp.close()
elif maintype == "audio":
    print 'audio'
    fp = open(fileToSend, "rb")
    attachment = MIMEAudio(fp.read(), _subtype=subtype)
    fp.close()
else:
    print 'else'
    fp = open(fileToSend, "rb")
    attachment = MIMEBase(maintype, subtype)
    attachment.set_payload(fp.read())
    fp.close()
    encoders.encode_base64(attachment)
attachment.add_header("Content-Disposition", "attachment", filename=fileToSend)
msg.attach(attachment)

server = smtplib.SMTP("EMAIL-SERVER.DOMAIN.COM")
server.set_debuglevel(0)
server.sendmail(emailfrom, emailto, msg.as_string())
server.quit()

GIT: Switching from code branch to dev/master

We recently upgraded to a latest version of Django (1.8). Earlier, we were running Django 1.2 and we created a new GIT branch called django18 that had all the migration changes. After going Live, we planned to stay on the django18 branch for a while so that if anything goes catastrophic, we can revert back to dev/master. But it had been a few months now and we didn’t revert back to dev and master. After I merged the django18 branch with dev, and then merged dev to master – here’s what I did to switch our dev and production server to run dev and master branches respectively. It wasn’t as easy as doing a checkout dev/master and just pulling code. With a lot of code and structural changes, I had to run through a lot of hoops to get this done right:

# Make sure the Django1.8 branch is up to date
git checkout task/django18
git status

# update dev branch
git checkout dev
git pull varun.verma dev

# update master branch
git checkout master
git pull varun.verma master

'''
error: The following untracked working tree files would be overwritten by merge:
	portal/__init__.pyc
	portal/api/__init__.pyc
	portal/api/admin.pyc
	portal/api/management/__init__.pyc
	portal/api/migrations/0001_initial.pyc
	portal/api/migrations/__init__.pyc
	portal/api/models.pyc
	portal/api/urls.pyc
	...
Aborting
'''

# View the the untracked files that will be cleaned. Review that all are .pyc files
git clean -fx --dry-run

# Now remove the untracked files
git clean -fx

# pull master branch again and it will complain about the statuc files as shown below
git pull varun.verma master

'''
From gitserver.domain.com:/var/git/your_repo
 * branch            master     -> FETCH_HEAD
error: The following untracked working tree files would be overwritten by merge:
	portal/static/admin/css/base.css
	portal/static/admin/css/changelists.css
	portal/static/admin/css/dashboard.css
	portal/static/admin/css/forms.css
	portal/static/admin/css/ie.css
	...
Please move or remove them before you can merge.
Aborting
'''

# View the untracked directories
git clean -dfx --dry-run
'''
Would remove portal/bi/management/
Would remove portal/another_app/management/
Would remove portal/static/admin/
'''

# Now get rid of the portal/static directory and build it again later
git clean -dfx

# pull master branch again and it will pull this time and list the additions/deletions/modifications
git pull varun.verma master

# make sure master local and origin are in sync
git status

'''
On branch master
Your branch is up-to-date with 'origin/master'.
'''

# now pull the static files
python manage.py collectstatic --settings= portal.settings_stage

# modify wsgi.py file to use the stage settings file
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "portal.settings_stage")

# restart apache
apache2ctl restart

# You may notice a list of .pyc files that now show as modified. You will have to checkout each of them individually
git checkout portal/__init__.pyc
...

Git – rename a file

Here’s how I just renamed a file under a git repository since I didn’t want to lose tracking changes by just deleting the old file and adding the new one to the repository. Although I use SmartGit but I had to do the renaming part on the terminal.

1. Rename the file using git command

$ git mv old_filename new_filename

2. Use git status to check the old and new file names.

$ git status
# On branch your-branch
# Changes to be committed:
#   (use "git reset HEAD ..." to unstage)
#
#     renamed: old_filename -> new_filename
#

3. Commit the file using SmartGit or from terminal using

$ git commit -m "Rename file"

4. Push changes to the remote repository from SmartGit or from terminal using

$ git push

Installing / Re-Installing Apache, PHP, libapache2-mod-php

The following is done on an Ubuntu machine but hoping it should cover other Debian instances.

I accidentally deleted php7.0.conf and php7.0.load from the mods-available folder. These files come with libapache2-mod-php. I uninstalled the module and installed it again – but the files didn’t appear. A lot of blogs suggested to remove Apache, PHP and libapache2-mod-php altogether and installing it again and it didn’t work. I used remove, purge and everything suggested but didn’t help. Eventually, I had to search for the installed module by:

dpkg --get-selections | grep libapache2-mod-php

I installed the module above as is but the result showed me libapache2-mod-php7.0. So I went ahead and removed and purged the module listed after which I installed the module again (without the 7.0 suffix) and I was able to recover the files again.

apt-get remove --purge libapache2-mod-php7.0
apt-get install libapache2-mod-php

Since I am covering this topic, let’s talk about installing Apache on Debian and the supported PHP packages along with SSL support.

apt-get install apache2
apt-get install php
apt-get install php-curl
apt-get install libapache2-mod-php

Enable SSL on the site:

cd /etc/apache2/sites-enabled
ln -fs ../sites-available/default-ssl.conf 

Finally, add the necessary modules to support PHP/SSL on Apache

cd /etc/apache2/mods-enabled
ln -fs ../mods-available/ssl.conf
ln -fs ../mods-available/ssl.load 
ln -fs ../mods-available/socache_shmcb.load
ln -fs ../mods-available/php7.0.conf
ln -fs ../mods-available/php7.0.load

Do not forget to restart apache with

service apache2 restart

Python – Iterate over a list and check if it’s not Empty

I have a list that I wanted to iterate over. When I used a list comprehension, it would fail if the list was None. But it would if the list was empty [].

The old and easy way of doing this was:

if tags:
	for t in tags:
		# Do stuff with t

The pythonic way of merging the if statement within the list comprehension is:

for t in [t for t in (tags or [])]:
	# Do stuff with t

If you had a dictionary, you could use a ternary condition within list comprehension:

([k for k, v in tags] if tags is not None else [])

OR, You can embed the ternary condition in the comprehension as well:

[k for k, v in (tags if tags is not None else [])]

The original stack overflow post.