Ansible module for enabling/disabling auto-assign public IP on AWS subnets

A custom ansible module for modifying “Enable/Disable auto-assign public IP” attribute to automatically request a public IP address for instances launched into the provided subnet

To modify your subnet’s public IP addressing behavior through AWS console:

  • Open the Amazon VPC console at https://console.aws.amazon.com/vpc/.
  • In the navigation pane, choose Subnets.
  • Select your subnet, choose Subnet Actions, and then Modify Auto-Assign Public IP.
  • The Enable Auto-assign Public IP check box, if selected, requests a public IP address for all instances launched into the selected subnet. Select or clear the check box as required, and then choose Save.
  • Read this for more info

Or use below custom Ansible module:

Note: at the time of writing this post, there was no such option in ansible module, “ec2_vpc”, to modify auto-assign public attribute of a subnet

Convert UTC datetime object to epoch seconds

A timedelta object represents a duration, the difference between two dates or times.

Convert UTC time to PST time in python

I wrote a small function which converts a datetime object from a timezone to a new timezone.
file name : tz2ntz.py

In action:

You can see all available time zones using pytz.all_timezones

How to convert an epoch second to a python datetime object?

The Epoch second (POSIX time/Unix time) is the number of seconds that have elapsed since January 1, 1970. Epoch seconds are not counting leap seconds.

So how we can convert an epoch second to human readable time?

For that, first we need to create a datetime object for the day 1970/Jan/01

then put the epoch second which you would like to convert to a human readble format in a varaible

then create a timedelta object for the epoch_second

then add dobj1 to dobj2

or in short

AWK practical tips for parsing access logs

#1 Find top 10 IPs from an access log

Now you can pipe the output to a pipe and can find top 10 IPs using sort and head -10

#2 Suppose you have an access log which have timestamp in epoch seconds like following and you want to print hourly QPS count

Now you know how to get an hourly data and you know how to pass bash variable inside awk. So using a for loop, right you can get data for all hours separately

Notes:

  • I tested this in awk version 3.1.7. You can find your awk version using awk –version
  • Assuming log is getting rotated every day

Click me for the official page of AWK

ctime, mtime and atime – the Linux timestamps

Even though the timestamps are filesystems specific implementation, following are the main timestamps which all Linux filesystems have.

  • ctime – The ctime (change time) is the time when changes made to the file’s inode (owner, permissions, etc.). The ctime is also updated when the contents of a file change. You can view the ctime with the ls -lc command
  • atime – The atime (access time) is the time when the data of a file was last accessed. Displaying the contents of a file or executing a shell script will update a file’s atime, for example. You can view the atime with the ls -lu command
  • mtime – The mtime (modify time) is the time when the actual contents of a file was last modified. This is the time displayed in a long directory listing (ls -l)

For more clarity on timestamps:

Following are the system calls for retrieving information about a file

  • stat()
  • lstat()
  • fstat()

These system calls differ only in the way that file is passed. stat() returns information about the named file. lstat() is also doing the same but if the named file is a link, the information about the link itself will return rather than the file to which the link points. fstat() returns information about a file referred to by an open file descriptor.

The ext4 filesystem have implemented few more timestamps which are following:

  • dtimedeletion time
  • crtimecreation time

You can read more about ext4 timestamps in following link:

How to make google drive as your automated backup location using python

Before you going to coming paragraphs of this article, I recommend you to watch following youtube video so you will get what is going to describe here:

[http://www.youtube.com/watch?v=zJVCKvXtHtE]
Read following links to learn more about the authentication mechanism google API is using :
https://developers.google.com/accounts/docs/OAuth2
https://developers.google.com/api-client-library/python/guide/aaa_oauth

As Claudio (GOOGLE) mentioned in the above video, (http://youtu.be/zJVCKvXtHtE?t=12m18s), we need to use some logic (client library) to sort your crendentials for reusing. Other wise on every execution of ‘quickstart.py’ script, you need human interaction for getting tokens. I added 6 lines to google’s ‘quickstart.py’ to keep authorization code for reusing. So as long as the user has not revoked the access granted initially to the application, you don’t need a human interaction.

For more details : https://developers.google.com/accounts/docs/OAuth2InstalledApp#refresh

You can get modified google’s quickstart.py using following

or

Goto : https://github.com/sukujgrg/google_drive

So, you got the idea how to deal with Google API and authentication to Google drive without human interaction. Now, you just need to apply your modifications to the script, suitable to your environment and then just put a cron job.

Difference between $* and $@ in bash?

$* and $@, both these bash special variables expands to the positional parameters, starting from the first one.

These variables are same (expand positional parameters in same way) when using without double quotes. If these variables are using inside double quotes, it will expand positional parameters differently.

$* within double quotes ("$*") is equivalent to the list of positional parameters, separated by IFS variable.

Suppose IFS is ":" and hence expansion of "$*" will be like "$1:$2:$3:…"

And $@ within a pair of double quotes ("$@") is equivalent to the list of positional parameters separated by unquoted spaces, i.e., "$1" "$2".."$N". Or in other words, it is equivalent to the list of positional parameters where each parameters are double quoted.

For sake better understanding I wrote a script named star_and_at.sh and pushed to my public github repo

You can clone my bash github public repository directly using following command

IPv4 Subnetting

In this article I am explaining IPv4 subnetting in following two situations.

1. Subnetting when given a required number of networks [Example 1]
2. Subnetting when given a required number of hosts [Example 2]

Note : You need to have basics of IPv4 address types and basics of subnetting

Example 1 :

You have a Class C network range 210.20.1.0 and need to break it into 20 separate networks.

Note : Here you have the information about number of nerworks you needed.

Solution : 

Continue reading “IPv4 Subnetting”

Find out what all filesystems are supported by linux kernel

/proc is very special in that it is also a virtual filesystem. It’s sometimes referred to as a process information pseudo-file system. In /proc there is a file named filesystems. And as the name says, it contains what all filesystems that are supported by the running kernel.