Python – Unescape quotes from JSON string

I had a data dump that stored JSON data in escaped/serialized format. Here’s a sample:

escaped_json = {\"payload\":{\"0\":{\"k1\":\"v1\", \"k2\":\"v2\", \"k3\":\"v3\"}, \"1\":{\"k4\":\"v5\", \"k6\":\"v4\", \"k5\":\"v6\"}}}

I wanted to convert this into valid JSON that can be ready by any parser. I had an option to substitute \” with ” but then there might have been other chars in the string, escaped or serialized that would have caused trouble later on. So I found this useful way of unescaping the text:

json_string = escaped_json.decode('string_escape')

to return:

{"payload":{"0":{"k1":"v1", "k2":"v2", "k3":"v3"}, "1":{"k4":"v5", "k6":"v4", "k5":"v6"}}}

PHP – Include relative path

If the PHP script is executed from a shell script that isn’t residing in the same directory, the php include statement would fail as it would look for the include file within the directory where the shell script is executed from. Therefore, the usage of __DIR__ in the include statement is much more desired way of including files.

include_once __DIR__."/../dbConn.php";

Note that the __DIR__ path doesn’t include the trailing slash so you may want to append it in your own code if you’re gathering the file from a different location.

Sudo without password

Use the following line (in /etc/sudoers) to sudo without having the system to prompt without password.

# User privilege with no password
varun.verma     ALL=NOPASSWD:ALL

When multiple entries match for a user, they are applied in order. Where there are multiple matches, the last match is used (which is not necessarily the most specific match). Therefore, it is important that this statement must be the last line of the file or beyond the group permission.

Retrieving the last record in each group

This is what I had been looking for a long time now. I have a table where I accumulate analytical data about customers’ usage like storage, bandwidth, licensed users, etc. I get a daily report for each of the server. My goal was to get the most recent data for any customer/server and so I found a way to do that. Thanks to this post. In spite of bookmarking this link, I decided to make notes of how the query for the LEFT JOIN works to get the last set of data for a customer.

Here’s the query I would use to get the latest row for each hostname:

SELECT u.*
FROM usage_data u LEFT JOIN usage_data ut
ON (u.hostname = ut.hostname AND u.id < ut.id)
WHERE tt.id IS NULL
ORDER BY u.hostname

Let’s assume we have the following set of data for a customer/hostname – usage_data(id, hostname, report_date, storage, bandwidth, licensed_users ….).

id | hostname            | report_date | .....
1  | customer.domain.com | 03/15/2015  | .....
2  | customer.domain.com | 03/16/2015  | .....
3  | customer.domain.com | 03/17/2015  | .....

When I do a simple left join, ON (u.hostname = ut.hostname), I’ll see resultant data with following id mappings:
1-3
1-2
1-1
2-3
2-2
2-1
3-3
3-2
3-1

But when I add another condition, the join looks like ON (u.hostname = ut.hostname AND u.id < ut.id), I'm only mapping rows for which u.id < ut.id. Therefore, you get the following mappings for id in [1, 2]
1-3
1-2
2-3
Now, for the id=3 in table u, there isn't a row in ut that satisfies the condition u.id < ut.id. But since we're using a LEFT JOIN, the database engine ensures that there is an entry for each row on the LEFT table. Therefore, the engine returns row for id=3 in table u and replaces by NULL for each value in table ut so our result set looks like the following
1-3
1-2
2-3
3-NULL

That's what we wanted. A way to identify a row uniquely that is the latest. Now if we add another WHERE clause in the result set, to get only that data that has ut.id as NULL, we will get the row corresponding to the latest id value in table u. Therefore, the LEFT JOIN query makes a little more sense to me now and hopefully to you as well.