AWK Tips

August 26th, 2014 No comments

AWK one liners.

# Find top 10 IPs from an access log
awk '{ array[$2]++ } END {for (ip in array) print array[ip],ip}' access.log | sort -nr | head -10

Click me for the official page of AWK

Categories: Bash, Linux, Scripting

ctime, mtime and atime – the Linux timestamps

August 26th, 2014 No comments

Even though the timestamps are filesystems specific implementation, following are the main timestamps which all Linux filesystems have.

  • ctime – The ctime (change time) is the time when changes made to the file’s inode (owner, permissions, etc.). The ctime is also updated when the contents of a file change. You can view the ctime with the ls -lc command
  • atime – The atime (access time) is the time when the data of a file was last accessed. Displaying the contents of a file or executing a shell script will update a file’s atime, for example. You can view the atime with the ls -lu command
  • mtime – The mtime (modify time) is the time when the actual contents of a file was last modified. This is the time displayed in a long directory listing (ls -l)

For more clarity on timestamps:

cat file # file's atime is updated

chmod 755 file # file's ctime is updated

echo "new contents" >> file # file's ctime and mtime are updated

vi file # if you add/delete some lines ctime and mtime will get updated

Following are the system calls for retrieving information about a file

  • stat()
  • lstat()
  • fstat()

These system calls differ only in the way that file is passed. stat() returns information about the named file. lstat() is also doing the same but if the named file is a link, the information about the link itself will return rather than the file to which the link points. fstat() returns information about a file referred to by an open file descriptor.

The ext4 filesystem have implemented few more timestamps which are following:

  • dtime – deletion time
  • crtime – creation time

You can read more about ext4 timestamps in following link:

Categories: Linux

How to make google drive as your automated backup location using python

December 30th, 2012 Comments off

Before you going to coming paragraphs of this article, I recommend you to watch following youtube video so you will get what is going to describe here:


Read following links to learn more about the authentication mechanism google API is using :
https://developers.google.com/accounts/docs/OAuth2
https://developers.google.com/api-client-library/python/guide/aaa_oauth

As Claudio (GOOGLE) mentioned in the above video, (http://youtu.be/zJVCKvXtHtE?t=12m18s), we need to use some logic (client library) to sort your crendentials for reusing. Other wise on every execution of ‘quickstart.py’ script, you need human interaction for getting tokens. I added 6 lines to google’s ‘quickstart.py’ to keep authorization code for reusing. So as long as the user has not revoked the access granted initially to the application, you don’t need a human interaction.

For more details : https://developers.google.com/accounts/docs/OAuth2InstalledApp#refresh

You can get modified google’s quickstart.py using following:

git clone git@github.com:sukujgrg/google_drive.git

or
Goto : https://github.com/sukujgrg/google_drive

So, you got the idea how to deal with Google API and authentication to Google drive without human interaction. Now, you just need to apply your modifications to the script, suitable to your environment and then just put a cron job.

Categories: Python, Scripting

How to use memcached with python and MySQL

December 27th, 2012 Comments off

memcached_banner75memcached is an open source project designed to make use of the spare RAM in multiple servers to act as a cache for frequently accessed bits of information. For example, consider a typical website; a web site served up dynamically will have certain components or information constant throughout the life of the page. Loading this information each time through a query to the database is very unnecessary. In this point the importance of caching is coming to the picture. If you have a caching mechanism on RAM rather than on a Hard Disk, you can imagine how fast it will be.

For demonstrating the performance of memcached powered application, I wrote a script called memcached_demo.py.

You can clone this script from my github repository using following:

git clone git@github.com:sukujgrg/mysql_memcached.git

Or, if you just want to understand the script use following link :
https://github.com/sukujgrg/mysql_memcached

What this script is doing:

Read more…

Convert currency values from command line

October 17th, 2012 Comments off

If you are in Linux Administration field, you must study Python. There are lot off discussions we can see in Internet about Python v/s Perl. I don’t want to talk anything about it. Python will help you to automate lot of stuffs in very easy way. Python have lot of modules, built in as well as external. I usually do automation using Bash. But bash have always certain limitations and so decided to study python. I wrote a small python script for converting currency values among different currencies. You can always download latest version of it from my GitHub account using following command.

git clone git@github.com:sukujgrg/currency_converter.git

Help menu:

sgeorge-mba:~ sgeorge$ currency_converter.py --help
usage: Example: currency_converter.py 1 USD INR

positional arguments:
  COUNT          how many count you want to convert
  CURRENCY       which CURRENCY (code) you want to convert from
  CURRENCY       which CURRENCY (code) you want to convert to

optional arguments:
  -h, --help     show this help message and exit
  -l, --list     print all available currency code
  -v, --version  show program's version number and exit

Usage examples:

sgeorge-mba:currency_converter sgeorge$ python currency_converter.py 1 EUR USD
1 EUR = 1.3097 USD
sgeorge-mba:currency_converter sgeorge$ python currency_converter.py 1 USD EUR
1 USD = 0.7635 EUR
sgeorge-mba:currency_converter sgeorge$ python currency_converter.py 2 USD EUR
2 USD = 1.5270 EUR
sgeorge-mba:currency_converter sgeorge$ python currency_converter.py 1 INR EUR
1 INR = 0.0144 EUR
sgeorge-mba:currency_converter sgeorge$ python currency_converter.py 1 INR USD
1 INR = 0.0189 USD
sgeorge-mba:currency_converter sgeorge$ python currency_converter.py 1 USD INR
1 USD = 52.9000 INR

Read more…

Categories: Linux, Python, Scripting

Difference between $* and $@ in bash?

June 26th, 2012 No comments

$* and $@, both these variables expands to the positional parameters, starting from the first one.

These variables are same (expand positional parameters in same way) when using without double quotes. If these variables are using inside double qoutes, will expand positional parameters differently.

Read more…

Categories: Bash, Linux, Scripting

Subnetting

December 14th, 2011 No comments

In this article I am explaining IPv4 subnetting in following two situations.

1. Subnetting when given a required number of networks [Example 1]
2. Subnetting when given a required number of hosts [Example 2]

Note : You need to have basics of IPv4 address types and basics of subnetting

Example 1 :

You have a Class C network range 210.20.1.0 and need to break it into 20 separate networks.

Note : Here you have the information about number of nerworks you needed.

Solution : 

Read more…

Categories: Tutorials

TCP/IP network model

November 29th, 2011 No comments

In this article I am explaining the Encapsulation part of a TCP/IP network model.

The process of sending data can be viewed as a five-step process in TCP/IP model.

Note: For understanding the things explained in this article you need to have a good idea about TCP/IP network model.

Step 1:
Create and encapsulate the application data with any required application layer headers.
For example, the HTTP OK message can be returned in an HTTP header, followed by part of the contents of a web.

Read more…

Categories: Tutorials

Find out what all filesystems are supported by linux kernel

November 25th, 2011 No comments

“/proc” contains many files critical to a Linux OS. In “/proc” there is a file named “filesystems”. And as the name implies, it contains what all filesystems are supported by a running kernel.

Read more…

Categories: Linux

“test” operators in Bash

November 25th, 2011 No comments

Bash is the GNU Project’s shell. Bash is the Bourne Again SHell. Bash is an sh-compatible shell that incorporates useful features from the Korn shell (ksh) and C shell (csh). It is intended to conform to the IEEE POSIX P1003.2/ISO 9945.2 Shell and Tools standard. It offers functional improvements over sh for both programming and interactive use. In addition, most sh scripts can be run by Bash without modification.

Click on me for the official page of BASH

Here I am trying to list almost all available bash “tests” (file tests, string tests, arithmetic tests)

File tests Read more…

Categories: Bash, Scripting