Friday, July 31, 2015

Python Eventlet - Examples

Python Eventlets - Snippets

Python Subprocess - Run Command in Background

Python : Read JSON

RabbitMQ and AMQP

Learn RabbitMQ

CURL / File Upload / Update Database / PHP

 curl -i -F userfile=@orders1.png http://www.some-domain.in/giridharmb/upload.php
 

upload.php

<?php

    $uploaddir = '/home/user123/public_html/giridharmb/uploads/';

    $uploadfile = $uploaddir . basename($_FILES['userfile']['name']);
    

    $operation_successful = array("upload" => "succcessful");
    $operation_not_successful = array("upload" => "failed");

    if (move_uploaded_file($_FILES['userfile']['tmp_name'], $uploadfile))
    {
        echo json_encode($operation_successful);
    }
    else
    {
       echo json_encode($operation_not_successful);
    }
?>

Fetch Data from DB

curl -X GET "http://some-domain.in/giridharmb/fetch_mysql_json.php”

[{"varname":"dal4.con1.cpuload","varvalue":"55"},{"varname":"dal4.con2.cpuload","varvalue":"55"},{"varname":"dal4.con555.cpuload","varvalue":"55"},{"varname":"dal5","varvalue":"59"},{"varname":"dal5.con555.cpuload","varvalue":"55”}]

Push Data into DB with values passed via CURL

curl -X GET "http://some-domain.in/giridharmb/push_into_mysql.php?variable=dal5.con555.cpuload&value=55”

Output:

{"operation":"successful”}

push_into_mysql.php


<?php

    $varname = $_GET["variable"];
    $value = $_GET["value"];

    $my_return_array = array();

    if( empty($varname) == TRUE )
    {
        $my_return_array = array("operation" => "failed");
        echo json_encode($my_return_array);
        return;
    }

    if( empty($value) == TRUE )
    {
        $my_return_array = array("operation" => "failed");
        echo json_encode($my_return_array);
        return;
    }

    $servername = "localhost";
    $username = "dbuser123";
    $password = "dbpassword123";
    $dbname = "mydb1";

    // Create connection
    mysql_connect($servername, $username, $password ) or die(mysql_error());

    mysql_select_db($dbname) or die(mysql_error());


    $my_query = "INSERT INTO keyvalue (varname,varvalue) VALUES('" . $varname . "','" . $value . "') ON DUPLICATE KEY UPDATE varvalue='" . $value . "'";

    $retval = mysql_query( $my_query );


    if(! $retval )
    {
        $my_return_array = array("operation" => "failed");
    }
    else
    {
        $my_return_array = array("operation" => "successful");
    }
    
    mysql_close($conn);

    echo json_encode($my_return_array);
    
?>  

File Upload

curl -i -F userfile=@ceph.osd.tree.json http://server1.company.com/fups/up.php

Output


HTTP/1.1 100 Continue

HTTP/1.1 200 OK
Date: Tue, 07 Apr 2015 09:21:02 GMT
Server: Apache/2.4.7 (Ubuntu)
X-Powered-By: PHP/5.5.9-1ubuntu4.3
Vary: Accept-Encoding
Content-Length: 335
Content-Type: text/html

<p>File is valid, and was successfully uploaded.
</p><pre>Here is some more debugging info:Array
(
[userfile] => Array
(
[name] => ceph.osd.tree.json
[type] => application/octet-stream
[tmp_name] => /tmp/phpbNXqb8
[error] => 0
[size] => 26145
)

)

up.php


<?php
$uploaddir = '/var/www/html/fups/data/';
$uploadfile = $uploaddir . basename($_FILES['userfile']['name']);

echo "<p>";

if (move_uploaded_file($_FILES['userfile']['tmp_name'], $uploadfile))
{
echo "File is valid, and was successfully uploaded.\n";
}
else
{
echo "Upload failed";
}

echo "</p>";
echo '<pre>';
echo 'Here is some more debugging info:';
print_r($_FILES);
print "</pre>";
?>


Wednesday, July 29, 2015

BASH Arrays


BASH ARRAYS

To declare an array in bash

Declare and an array called array and assign three values:

array=( one two three )

More examples:

files=( "/etc/passwd" "/etc/group" "/etc/hosts" )
limits=( 10, 20, 26, 39, 48)
To print an array use:

printf "%s\n" "${array[@]}"
printf "%s\n" "${files[@]}"
printf "%s\n" "${limits[@]}"

To Iterate Through Array Values

Use for loop syntax as follows:

for i in "${arrayName[@]}"
do
   # do whatever on $i
done

$i will hold each item in an array. Here is a sample working script:

#!/bin/bash
# declare an array called array and define 3 vales
array=( one two three )
for i in "${array[@]}"
do
        echo $i
done

Python Networking Programming

Java Interview Questions

Linux Performance Analysis and Tools

RedHat: Virtualization & libvirt

OpenStack Swift Configuration Reference

OpenStack & Python API

OpenStack Request Flow

OpenStack Networking Concepts

OpenStack Operations Manual

OpenStack Networking Basics

OpenStack Networking 101

OpenStack Network Internals

OpenStack ML2

OpenStack Admin Guide

OpenStack : Provider Networks with Linux Bridge

OpenStack Docs : Provider Networks with OpenVSwitch

OpenStack Neutron & OpenVSwitch Troubleshooting

OpenStack Neutron

CLI Reference Guide

OpenStack CINDER (Juno) Config Reference

OpenStack RPC Installation Guide

Basic Introduction of OpenVSwitch

OpenStack Bootcamp

OpenStack Request Flow

OpenStack Networking

OpenStack Ports

CIDR & Netmask

Internet domain socket states

NETSTAT : Examples

TCP-vs-UDP

SSL Handshake : Part 3

SSL Handshake : Part 2

SSL Handshake : Part 1

Network Stack

TCP Handshake

TCP Explained

Understanding TCP-IP

Tuesday, July 28, 2015

Graphite and StatsD - for Metrics/Analytics

HTTP : GET/PUT/POST/JSON



Linux provides a nice little command which makes our lives a lot easier.

GET:

with JSON:

curl -i -H "Accept: application/json" -H "Content-Type: application/json" -X GET http://hostname/resource

with XML:

curl -H "Accept: application/xml" -H "Content-Type: application/xml" -X GET http://hostname/resource


POST:

For posting data:

curl --data "param1=value1&param2=value2" http://hostname/resource


For file upload:

curl --form "fileupload=@filename.txt" http://hostname/resource

RESTful HTTP Post:

curl -X POST -d @filename http://hostname/resource

For logging into a site (auth):

curl -d "username=admin&password=admin&submit=Login" --dump-header headers http://localhost/Login

curl -L -b headers http://localhost/


GET approach with JSON result
curl -i -H "Accept: application/json" http://someHostName/someEndpoint | grep }| python -mjson.tool 

POST approach with JSON result

curl -X POST  -H "Accept: Application/json" -H "Content-Type: application/json" http://someHostName/someEndpoint -d '{"id":"IDVALUE","name":"Mike"}' | grep }| python -mjson.tool


Posting Data with Curl

curl --data 'foo=bar' \
        --trace-ascii /dev/stdout \
        http://edoceo.com/

Posting JSON Data with Curl

curl --data '{"id":"value", ""}' http://


Inspect SSL Certificates

Using curl we can also view information about the SSL certificates from the server. This also helps to diagnose if there are issues with the certificate chain.

~ $ curl -v https://edoceo.com/



Post STDOUT to Server via curl

Here we take the output of lshw and send that up to the Linux Hardware Database.

~ $ lshw -json |  curl -d@- -qs "http://edoceo-demo.com/upload"
~ $ lshw -json |  curl --data-ascii @- -qs "http://edoceo-demo.com/upload"
~ $ lshw -json |  curl --data-binary @- -qs "http://edoceo-demo.com/upload"

Tracing

Use the --trace-ascii option.

curl http://edoceo.com/ -d "hello=there" --trace-ascii /dev/stdout



Enables a full trace dump of all incoming and outgoing data, including descriptive informa‐ tion, to the given output file. Use "-" as filename to have the output sent to stdout.
This is very similar to --trace, but leaves out the hex part and only shows the ASCII part of the dump. It makes smaller output that might be easier to read for untrained humans.
This option overrides previous uses of -v, --verbose or --trace.
If this option is used several times, the last one will be used.

== Info: Adding handle: conn: 0x14a8880
== Info: Adding handle: send: 0
== Info: Adding handle: recv: 0
== Info: Curl_addHandleToPipeline: length: 1
== Info: - Conn 0 (0x14a8880) send_pipe: 1, recv_pipe: 0
== Info: About to connect() to edoceo.com port 80 (#0)
== Info:   Trying 74.207.248.164...
== Info: Connected to edoceo.com (74.207.248.164) port 80 (#0)
=> Send header, 74 bytes (0x4a)
0000: GET / HTTP/1.1
0010: User-Agent: curl/7.33.0
0029: Host: edoceo.com
003b: Accept: */*
0048: 
<= Recv header, 17 bytes (0x11)
0000: HTTP/1.1 200 OK
<= Recv header, 37 bytes (0x25)
0000: Date: Thu, 09 Jan 2014 02:05:51 GMT
== Info: Server Apache is not blacklisted
<= Recv header, 16 bytes (0x10)
0000: Server: Apache
<= Recv header, 55 bytes (0x37)
0000: Set-Cookie: edoceo=edla1uattv6kbep8ofqp3q5ri6; path=/
<= Recv header, 40 bytes (0x28)
0000: Expires: Thu, 19 Nov 1981 08:52:00 GMT
<= Recv header, 79 bytes (0x4f)
0000: Cache-Control: must-revalidate, no-cache, no-store, post-check=0
0040: , pre-check=0
<= Recv header, 22 bytes (0x16)
0000: Content-Language: en
<= Recv header, 23 bytes (0x17)
0000: Vary: Accept-Encoding
<= Recv header, 28 bytes (0x1c)
0000: Transfer-Encoding: chunked
<= Recv header, 40 bytes (0x28)
0000: Content-Type: text/html; charset=utf-8
<= Recv header, 2 bytes (0x2)
0000: 
<= Recv data, 1089 bytes (0x441)

HTTP Proxy over SSH

On the node which cannot access the internet:

(1)

ssh -f -N -n -L 8080:127.0.0.1:80 root@172.28.136.5

(2)

export http_proxy=http://localhost:8080/


(172.28.136.5 = node which has internet access)

SSH Tunnel on Mac OSX / Linux



SSH Tunnel in 30 Seconds (Mac OSX & Linux)

Some days, I wonder why VPN’s are really necessary when we can just use an SSH tunnel.

If you’re on Mac or a flavour of Linux, this SSH tunnelling tutorial is for you.

“A secure shell (SSH) tunnel consists of an encrypted tunnel created through a SSHprotocol connection. Users may set up SSH tunnels to transfer unencrypted traffic over a network through an encrypted channel.” – Wikipedia

Launch an SSH tunnel

To initiate your SSH tunnel, simply [open Mac OSX / Linux Terminal][1] and connect to your remote server via SSH with the following flags:

ssh -D 8080 -C -N username@example.com


This will launch our SSH tunnel on port 8080 and route all traffic (securely) through the server at example.com.

Browse the Web with Your SSH Tunnel (Chrome)

Now, let’s start browsing the web using our new SSH tunnel.
Mac OSX:
  1. Open Google Chrome
  2. Select ‘Chrome’ up the top left
  3. Select ‘Preferences’
  4. Select ‘Show advanced settings…’
  5. Select ‘Change proxy settings…’
  6. Select ‘SOCKS Proxy’
  7. Enter ’127.0.0.1′
  8. Enter port ’8080′
  9. Save changes by selecting ‘OK’
Fedora Linux:
  1. Open Google Chrome
  2. Select the wrench icon on the top right
  3. Select ‘Settings’
  4. Select ‘Show advanced settings…’
  5. Select ‘Change proxy settings…’
  6. Select ‘SOCKS Proxy’
  7. Enter ’127.0.0.1′
  8. Enter port ’8080′
  9. Save changes by selecting ‘OK’
Search Google for ‘my ip’ and take a look at what your IP address is now. Cool right? 

Exiting the SSH Tunnel

To exit the SSH tunnel, simply disable the SOCKS proxy within your browser.

TCP Dump Usage / Examples

VirtualBox - NAT Networking

GIT Cheat Sheet

GIT Tutorial

GIT - Simple Guide

GIT Submodules

Sublime : RSUB - Remote Sublime Edit

Sublime RSUB

--1--

Install rsub package through package manage : https://packagecontrol.io/installation

--2--

printf "Host *\nRemoteForward 52698 127.0.0.1:52698" >> ~/.ssh/config

--3--

sudo wget -O /usr/local/bin/subl https://raw.github.com/aurora/rmate/master/rmate;

--4--

sudo chmod +x /usr/local/bin/subl

Sublime Keyboard Shortcuts



Sublime Text 2 - Useful Shortcuts

  • super+r: go to methods
  • super+l: select line (repeat select next lines)
  • ctrl+shift+m: select content into brackets
  • super+shift+enter: insert line before
  • super+enter: inter line after
  • ctrl+shift+k: delete line
  • super+k-k: delete from cursor to end of line
  • super+k-backspace: delete from cursor to start of line
  • super+j: join lines
  • super+k-u: upper case
  • super+k-l: lower case
  • super+/: comment
  • ctrl+m: jump to matching brackets
  • super+alt+f: replace
  • super+ctrl+g: select all occurrences of current word for multiple editing
  • super+f2: Toggle bookmark
  • f2: next bookmark
  • shift+f2: previous bookmark
  • super+shift+f2: clear bookmarks


Tested in Mac OS X: super == command
Open/Goto

  • super+t: go to file
  • super+ctrl+p: go to project
  • super+r: go to methods
  • super+shift+p: command prompt
  • ctrl+g: go to line
  • ctrl+`: Python console
Editing

  • super+l: select line (repeat select next lines)
  • super+d: select word (repeat select others occurrences in context for multiple editing)
  • ctrl+shift+m: select content into brackets
  • super+shift+enter: insert line before
  • super+enter: inter line after
  • ctrl+shift+k: delete line
  • super+k-k: delete from cursor to end of line
  • super+k-backspace: delete from cursor to start of line
  • super+shift+d: duplicate line(s)
  • super+j: join lines
  • super+k-u: upper case
  • super+k-l: lower case
  • super+/: comment
  • super+alt+/: block comment
  • super+y: redo or repeat
  • super+shift+z: past and ident
  • ctrl+space: autocomplete (repeat to select next suggestion)
  • ctrl+m: jump to matching brackets
XML/HTML

  • super+shift+a: select content into tag
  • super+alt+.: close tag
Find/Replace

  • super+f: find
  • super+alt+f: replace
  • super+alt+g: find next occurrence of current word
  • super+ctrl+g: select all occurrences of current word for multiple editing
  • super+shift+f: find in files
Splits/Tabs

  • super+alt+1: Single column
  • super+alt+2: Two columns
  • super+alt+5: Grid (4 groups)
  • ctrl+[1,2,3,4]: Focus group
  • ctrl+shift[1,2,3,4]: Move file to group
  • super+[1,2,3...] Select tab
Bookmarks

  • super+f2: Toggle bookmark
  • f2: next bookmark
  • shift+f2: previous bookmark
  • super+shift+f2: clear bookmarks
Marks

  • super+k-space: set mark // super+k-; for me
  • super+k-w: delete from cursor to mark
  • super+k-a: select from cursor to mark
  • super+k-g: clear mark

VIM Short Cuts



KEYBOARD SHORTCUTS

w = Move to next word

W = Move to next blank delimited word

o = Open a new line after current line

O = Open a new line before current line

w = Move to next word

W = Move to next blank delimited word

e = Move to the end of the word

E = Move to the end of Blank delimited word

G = end of file

H = top of file

d$ = delete from cursor to end of line

dd = delete entire line

x,yd = delete lines from line-n to line-m

dw = delete current word

/string = search forward for a string

?string = search backward for a string

n = Search for next instance of strin

N = Search for previous instance of string

:set ic = Ignores case when searching

:set noic = Pays attention to case when searching

:n,ms/str1/str2/gc = Searches from n to m for str1; replaces str1 to str2

:g/str1/s/str2/str3/gc = Finds the line containing str1, replaces str2 with str3

:! cmd = execute ‘cmd’ shell command

Review of my Triumph Daytona 675R

Date of writing this review : Roughly around Dec / 2014

My Review on Daytona 675 R:

FYI : this is just  my opinion , you are more than welcome to have different thoughts about the bike :)

Bikes that I currently own: Yamaha RX135, Bajaj Pulsar 220, Ninja 300 and Daytona 675-R.

Review:
First of all , I love the looks of the bike - it's just too good. Secondly , the build quality is amazing. The fit and finish is second to none.  I compared this build quality / finish with a few litre class bikes and I must admit , the finish is way way way better !! I have the 675 R which has carbon fibre cockpit and front and rear mud flaps - which not only looks good , but also is strong. Thirdly , I love the whistling sound it makes each time it is revved up and down. It is so god damn addictive. The red line on the tacho meter starts at 15,000 RPM and ends at 18,000 RPM. Wow ! Next on the list which 675 R has is adjustable OHLINS suspension - apart from adjusting the front and rear pre-load (which you normally don't change) , we can play around with compression and rebound settings - depending on the the kind of road you are riding on (bumpy or has pot holes or smooth etc) - or if it's a race track. It's awesome feeling when you can completely tune it based on the feedback you want to get off the road. Coming to the power - 126 BHP is quite a bit for middle weight super sports, and it's got quite a lot of mid range torque. The feeling of its acceleration when you open the throttle low down the Rev range is awesome. When you get past 8000 rpm , the valves fully open and the sound goes Ballistic ! The stock tyres are Pirelli super corsa SP - which are race specs. It's mostly meant for race track. On dry roads, excellent grip. Wet roads , I'm not confident. That's pretty much expected.  If you are planning to take it to the track not too often but mostly on city roads and highways , I would recommend you take off the stock Pirelli (which are expensive - roughly 30k for both front and back) , save it for track use and put Michelin pilot road 2 or something like that. Front is 120 size and back is 180 size . Apart from these, it has the steering damped. Oh I forgot to mention the amazing QUICK SHIFTER. While riding , you can upshift (not applicable for down shift) without applying the clutch and while the throttle is wide open !! On the race track , it's amazing ! You can save valuable time for quicker lap times. On open highways , when you use the quick shifter , it's takes very less time to get to 150 km/hr. My top speed so far has been 245 km/hr. Could have gone faster , but did not. But if you are a seasoned racer riding a bike without a quick shifter  , you would be behind a bike with a quick shifter by a negligible margin - but it's fun to use. Warning - make sure you use the quick shifter when the engine rpm is above 5000. If you are below - it might screw up the gearing. This is triumph dealership recommendations , not mine. Triumph engineers are best persons to answer this. Inside the city , if I ride aggressively , mileage can come down to as low as 12 km per litre . If I ride economically inside the city , I can get 16 km per litre. On highways - on an average I can get 19/20 km per litre. Tank capacity is 17.2 L. Octane 91 or above is recommended fuel but not a must. But you can fill petrol from any bunk , just make sure it's known to give good quality petrol. I usually fill SHELL normal or SHELL super. Many times , BP Speed 97 (octane 97). Engine is so smooth even when it is revved hard. No vibrations felt on fairings or front wind shield or mirrors or seat. In jam packed traffic - you can feel so much heat near the thighs. But that's normal. On highways , when you are riding fast , you won't feel it because of continuous in flow of wind and temperature of engine is much lower on highways. It also has programmable shift lights - the blue LEDs can be programmed as to at what rpm you want them on - like at 10,500 RPM one blue led comes on , then at 10,600 another led comes on (so in total 2 are on) , then at 10,700 one more led comes on (3 in total are on) ... Etc so at 11,000 rpm , all of them are on and keeps flashing on and off together to make it visually evident that you need to shift up a gear. All this is completely programmable. It also has a lap timer. Also shows mileage (km/L) on the console apart from engine temperature , average speed , trip meters , clock , ABS (on/off) etc. front brakes are dual disc - I love the braking on this bike ! Rear is ok types , front is excellent. Unlike Japanese bikes , the service interval is every 10,000 KMS or one year - which ever comes first. Coming to accessories - there are lots ! Visit your nearest triumph dealership and you will know. If they don't have it - visit the triumph UK Website , and see what you want and tell the dealership to get it for you - they would be happy to get it for you , down side - waiting time :) ! I would highly recommend getting frame sliders and engine crash protectors (for gear box and clutch plate). Go for zero dep insurance. Coming to riding position - it's damn aggressive ! The riding triangle is Even more aggressive than Kawasaki ZX 10R ! I have a ninja 300 also. But comparing to 300 , when I sat on 675R for the first time , I felt like my hands were resting on the ground and some one had lifted both my legs high up in the air ! I have already done 12,000 Kms on this under 5 months. (On ninja it was 25,000 kms under 10 months) - on the daytona I did not have any back problems for riding long distances  , but initially I had pain in the wrists , coz upper body is resting on the hands , but over time I got used to it - and now I ride it almost every day. No issues. Love it like crazy ! This bike is totally intoxicating ! Worth it's price. I did not even test ride it before I got it. I knew what triumph was and is. My opinion has not changed till date. What a bike this is !! 

Hope my review was useful ?

Check out my bike ride videos on youtube, my Youtube video channel:

Data Structures Interview Questions

A Brief Tutorial on CHEF

CHEF Fundamentals

CEPH RBD / User Management / Authentication

Links:
http://ceph.com/docs/master/rbd/rbd-openstack/
http://ceph.com/docs/master/rados/operations/user-management/
http://ceph.com/docs/master/man/8/ceph-authtool/
http://docs.openstack.org/juno/config-reference/content/ceph-backup-driver.html

sed -e 's/cinder-backup/cinder/g' -i /etc/cinder/cinder.conf

service cinder-volume restart;service cinder-scheduler restart;service cinder-backup restart;

ceph osd pool create backups 8192 8192


Default:

client.cinder
    key: AQASIpJVGEKOFxAAwR4pryr5/yrq5/BJFGIK8g==
    caps: [mon] allow r
    caps: [osd] allow class-read object_prefix rbd_children,allow rwx pool=volumes,allow rwx pool=vms, allow rx pool=images

ceph auth caps client.cinder mon 'allow r' osd 'allow class-read object_prefix rbd_children, allow rwx pool=volumes, allow rwx pool=vms, allow rx pool=images,allow rwx pool=backups'


# Ceph Cinder Integration (These are the Default Values)

volume_driver = cinder.volume.drivers.rbd.RBDDriver
rbd_pool = volumes
rbd_ceph_conf = /etc/ceph/ceph.conf
rbd_flatten_volume_from_snapshot = false
rbd_max_clone_depth = 5
rbd_store_chunk_size = 4
rados_connect_timeout = -1
glance_api_version = 2
rbd_user = cinder
rbd_secret_uuid = 95ed3580-fcb4-47b7-805a-fedd3e390077


backup_driver = cinder.backup.drivers.ceph
backup_ceph_conf = /etc/ceph/ceph.conf
backup_ceph_user = cinder
backup_ceph_chunk_size = 134217728
backup_ceph_pool = backups
backup_ceph_stripe_unit = 0
backup_ceph_stripe_count = 0
restore_discard_excess_bytes = true


Testing:

root@dal-appblx079-01_cinder_api_container-fbc3bbc5:~# cinder create 2 --display-name test123

+---------------------+--------------------------------------+
|       Property      |                Value                 |
+---------------------+--------------------------------------+
|     attachments     |                  []                  |
|  availability_zone  |                 nova                 |
|       bootable      |                false                 |
|      created_at     |      2015-07-15T06:01:42.836665      |
| display_description |                 None                 |
|     display_name    |               test123                |
|      encrypted      |                False                 |
|          id         | 40491936-3f2b-44e4-92f9-a0f34c67fe12 |
|       metadata      |                  {}                  |
|         size        |                  2                   |
|     snapshot_id     |                 None                 |
|     source_volid    |                 None                 |
|        status       |               creating               |
|     volume_type     |                 None                 |
+---------------------+--------------------------------------+

root@dal-appblx079-01_cinder_api_container-fbc3bbc5:~# cinder backup-create test123

+-----------+--------------------------------------+
|  Property |                Value                 |
+-----------+--------------------------------------+
|     id    | cceccf64-4643-4e0b-bafe-18d59f7fe79d |
|    name   |                 None                 |
| volume_id | 40491936-3f2b-44e4-92f9-a0f34c67fe12 |
+-----------+--------------------------------------+

root@dal-appblx079-01_cinder_api_container-fbc3bbc5:~# cinder backup-list

+--------------------------------------+--------------------------------------+-----------+------+------+--------------+-----------+
|                  ID                  |              Volume ID               |   Status  | Name | Size | Object Count | Container |
+--------------------------------------+--------------------------------------+-----------+------+------+--------------+-----------+
| 1cde0104-603a-483f-8794-a29f1f69d1b8 | a593b9bc-f9c1-4a34-8e33-69cafe094140 | available | None |  1   |     None     |  backups  |
| cceccf64-4643-4e0b-bafe-18d59f7fe79d | 40491936-3f2b-44e4-92f9-a0f34c67fe12 | available | None |  2   |     None     |  backups  |
+--------------------------------------+--------------------------------------+-----------+------+------+--------------+-----------+

CEPH : Block Devices & OpenStack

How Data Is Stored On CEPH Cluster

Ansible Tutorial

Ansible Quick Start

Ansible - stdout / Output Lines

Ansibe - Tips & Tricks

Ansible Vault

Introducing Ansible Vault

Ansible 1.5, which will release in a few weeks, adds a new command-line tool “ansible-vault”, and a new /usr/bin/ansible and/usr/bin/ansible-playbook option, “--ask-vault-pass”.

The idea here is pretty simple -- there is often a need to keep in configuration files, for use in playbooks and templates, certain data that you don’t want to expose in source control.

To give credit where credit is due, this feature isn’t *exactly* a new idea. Chef has a feature called “encrypted data bags”, for instance, though “vault” adds Ansible’s own flavor to it.

To do this, instead of opening your favorite editor, run the following command, which will launch the editor defined by your $EDITOR, or will default to vim if this is not set:

ansible-vault create vars.yml

The tool will ask you for a password to encrypt the file with. To edit it again later:

ansible-vault edit vars.yml

And to run a playbook that uses encrypted data:

ansible-playbook site.yml --ask-vault-pass

Should you get the vault password wrong, you’ll get a friendly error message.

What can be encrypted? Lots of things. group_vars and host_vars files, vars_files, things included with “include_vars”, and even individual playbooks or task files. Basically everything that is YAML in Ansible can be used with ansible-vault. It’s really generic.

There are a few extra commands. Suppose you have a vault-encrypted file and want to change the password?

ansible-vault rekey vars.yml

Or if you want to encrypt an existing plaintext file?

ansible-vault encrypt vars.yml

Or to permanently decrypt an existing file?

ansible-vault decrypt vars.yml

If you want to encrypt, decrypt, or rekey multiple files at the same time, you can do this as follows:

ansible-vault [encrypt|decrypt|rekey] vars1.yml vars2.yml vars3.yml

ansible.cfg

# config file for ansible -- http://ansible.com/
# ==============================================

# nearly all parameters can be overridden in ansible-playbook 
# or with command line flags. ansible will read ANSIBLE_CONFIG,
# ansible.cfg in the current working directory, .ansible.cfg in
# the home directory or /etc/ansible/ansible.cfg, whichever it
# finds first

[defaults]

# some basic default values...

inventory      = /etc/ansible/hosts
#library        = /usr/share/my_modules/
remote_tmp     = $HOME/.ansible/tmp
pattern        = *
forks          = 5
poll_interval  = 15
sudo_user      = root
#ask_sudo_pass = True
#ask_pass      = True
transport      = smart
#remote_port    = 22
module_lang    = C

# plays will gather facts by default, which contain information about
# the remote system.
#
# smart - gather by default, but don't regather if already gathered
# implicit - gather by default, turn off with gather_facts: False
# explicit - do not gather by default, must say gather_facts: True
gathering = implicit

# additional paths to search for roles in, colon separated
#roles_path    = /etc/ansible/roles

# uncomment this to disable SSH key host checking
#host_key_checking = False

# change this for alternative sudo implementations
sudo_exe = sudo

# what flags to pass to sudo
#sudo_flags = -H

# SSH timeout
timeout = 10

# default user to use for playbooks if user is not specified
# (/usr/bin/ansible will use current user as default)
#remote_user = root

# logging is off by default unless this path is defined
# if so defined, consider logrotate
#log_path = /var/log/ansible.log

# default module name for /usr/bin/ansible
#module_name = command

# use this shell for commands executed under sudo
# you may need to change this to bin/bash in rare instances
# if sudo is constrained
#executable = /bin/sh

# if inventory variables overlap, does the higher precedence one win
# or are hash values merged together?  The default is 'replace' but
# this can also be set to 'merge'.
#hash_behaviour = replace

# list any Jinja2 extensions to enable here:
#jinja2_extensions = jinja2.ext.do,jinja2.ext.i18n

# if set, always use this private key file for authentication, same as 
# if passing --private-key to ansible or ansible-playbook
#private_key_file = /path/to/file

# format of string {{ ansible_managed }} available within Jinja2 
# templates indicates to users editing templates files will be replaced.
# replacing {file}, {host} and {uid} and strftime codes with proper values.
ansible_managed = Ansible managed: {file} modified on %Y-%m-%d %H:%M:%S by {uid} on {host}

# by default, ansible-playbook will display "Skipping [host]" if it determines a task
# should not be run on a host.  Set this to "False" if you don't want to see these "Skipping" 
# messages. NOTE: the task header will still be shown regardless of whether or not the 
# task is skipped.
#display_skipped_hosts = True

# by default (as of 1.3), Ansible will raise errors when attempting to dereference 
# Jinja2 variables that are not set in templates or action lines. Uncomment this line
# to revert the behavior to pre-1.3.
#error_on_undefined_vars = False

# by default (as of 1.6), Ansible may display warnings based on the configuration of the
# system running ansible itself. This may include warnings about 3rd party packages or
# other conditions that should be resolved if possible.
# to disable these warnings, set the following value to False:
#system_warnings = True

# by default (as of 1.4), Ansible may display deprecation warnings for language
# features that should no longer be used and will be removed in future versions.
# to disable these warnings, set the following value to False:
#deprecation_warnings = True

# (as of 1.8), Ansible can optionally warn when usage of the shell and
# command module appear to be simplified by using a default Ansible module
# instead.  These warnings can be silenced by adjusting the following
# setting or adding warn=yes or warn=no to the end of the command line 
# parameter string.  This will for example suggest using the git module
# instead of shelling out to the git command.
# command_warnings = False


# set plugin path directories here, separate with colons
action_plugins     = /usr/share/ansible_plugins/action_plugins
callback_plugins   = /usr/share/ansible_plugins/callback_plugins
connection_plugins = /usr/share/ansible_plugins/connection_plugins
lookup_plugins     = /usr/share/ansible_plugins/lookup_plugins
vars_plugins       = /usr/share/ansible_plugins/vars_plugins
filter_plugins     = /usr/share/ansible_plugins/filter_plugins

# by default callbacks are not loaded for /bin/ansible, enable this if you
# want, for example, a notification or logging callback to also apply to 
# /bin/ansible runs
#bin_ansible_callbacks = False


# don't like cows?  that's unfortunate.
# set to 1 if you don't want cowsay support or export ANSIBLE_NOCOWS=1 
#nocows = 1

# don't like colors either?
# set to 1 if you don't want colors, or export ANSIBLE_NOCOLOR=1
#nocolor = 1

# the CA certificate path used for validating SSL certs. This path 
# should exist on the controlling node, not the target nodes
# common locations:
# RHEL/CentOS: /etc/pki/tls/certs/ca-bundle.crt
# Fedora     : /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem
# Ubuntu     : /usr/share/ca-certificates/cacert.org/cacert.org.crt
#ca_file_path = 

# the http user-agent string to use when fetching urls. Some web server
# operators block the default urllib user agent as it is frequently used
# by malicious attacks/scripts, so we set it to something unique to 
# avoid issues.
#http_user_agent = ansible-agent

# if set to a persistent type (not 'memory', for example 'redis') fact values
# from previous runs in Ansible will be stored.  This may be useful when
# wanting to use, for example, IP information from one group of servers
# without having to talk to them in the same playbook run to get their
# current IP information.
fact_caching = memory


# retry files
#retry_files_enabled = False
#retry_files_save_path = ~/.ansible-retry

[privilege_escalation]
#become=True
#become_method='sudo'
#become_user='root'
#become_ask_pass=False

[paramiko_connection]

# uncomment this line to cause the paramiko connection plugin to not record new host
# keys encountered.  Increases performance on new host additions.  Setting works independently of the
# host key checking setting above.
#record_host_keys=False

# by default, Ansible requests a pseudo-terminal for commands executed under sudo. Uncomment this
# line to disable this behaviour.
#pty=False

[ssh_connection]

# ssh arguments to use
# Leaving off ControlPersist will result in poor performance, so use 
# paramiko on older platforms rather than removing it
#ssh_args = -o ControlMaster=auto -o ControlPersist=60s

# The path to use for the ControlPath sockets. This defaults to
# "%(directory)s/ansible-ssh-%%h-%%p-%%r", however on some systems with
# very long hostnames or very long path names (caused by long user names or 
# deeply nested home directories) this can exceed the character limit on
# file socket names (108 characters for most platforms). In that case, you 
# may wish to shorten the string below.
# 
# Example: 
# control_path = %(directory)s/%%h-%%r
#control_path = %(directory)s/ansible-ssh-%%h-%%p-%%r

# Enabling pipelining reduces the number of SSH operations required to 
# execute a module on the remote server. This can result in a significant 
# performance improvement when enabled, however when using "sudo:" you must 
# first disable 'requiretty' in /etc/sudoers
#
# By default, this option is disabled to preserve compatibility with
# sudoers configurations that have requiretty (the default on many distros).
# 
#pipelining = False

# if True, make ansible use scp if the connection type is ssh 
# (default is sftp)
#scp_if_ssh = True

[accelerate]
accelerate_port = 5099
accelerate_timeout = 30
accelerate_connect_timeout = 5.0

# The daemon timeout is measured in minutes. This time is measured
# from the last activity to the accelerate daemon.
accelerate_daemon_timeout = 30 

# If set to yes, accelerate_multi_key will allow multiple
# private keys to be uploaded to it, though each user must
# have access to the system via SSH to add a new key. The default
# is "no".
#accelerate_multi_key = yes

Ansible - Check If File Exists

SUDO LDAP USER - Run Remote Command

ansible -i hosts servers-group-1 -m command -a "tail /var/log/syslog" -k -u gbhujanga -s -K
-k = --ask-password
-K = --ask-sudo-pass
-s = --sudo
[servers-group-1]
test-server.prod.walmart.com ansible_ssh_pass=root123 ansible_sudo_pass=root123

If ansible_ssh_pass & ansible_sudo_pass is mentioned in hosts files,then no need to mention -k and -K for ansible command.

Thursday, February 12, 2015

Use iPhone and Arduino to Control Home Appliances

https://www.youtube.com/watch?v=YYBSB4Fak4E



Code

File:get.php

<?php
    
    header("Access-Control-Allow-Origin: *");
    
    $param = $_GET['object'];

    $servername = "localhost";
    $username = "dbuser";
    $password = "dbpassword";
    $dbname = "myvardb";

    // Create connection
    mysql_connect($servername, $username, $password ) or die(mysql_error());

    mysql_select_db($dbname) or die(mysql_error());

    $result = mysql_query("SELECT varvalue FROM myvardb.keyvalue where varname = '" . $param . "'");

    $json = array();
    
    $total_records = mysql_num_rows($result);

    if($total_records >= 1){
      while ($row = mysql_fetch_array($result, MYSQL_ASSOC)){
        $json[] = $row;
      }
    }
    $ret_val = "{" . $json[0]["varvalue"] . "}";
    echo json_encode($ret_val);
?>

File:control.php

<!DOCTYPE html>
<html>
<head>
<meta name="viewport" content="width=device-width, initial-scale=1">
<link rel="stylesheet" href="http://code.jquery.com/mobile/1.4.5/jquery.mobile-1.4.5.min.css">
<script src="http://code.jquery.com/jquery-1.11.2.min.js"></script>
<script src="http://code.jquery.com/mobile/1.4.5/jquery.mobile-1.4.5.min.js"></script>
</head>
<body>

<div data-role="page" id="pageone">
  <div data-role="header">
    <h1>Home Appliance Controller</h1>
  </div>

  <div data-role="main" class="ui-content">
      
    <p>Lighting : F1L1</p>
    <a id="off_f1l1" href="#" class="ui-btn ui-btn-inline">OFF</a>
    <a id="on_f1l1" href="#" class="ui-btn ui-btn-inline">ON</a>
    <br/>
    <p>Geyser : F1G1</p>
    <a id="off_f1g1" href="#" class="ui-btn ui-btn-inline">OFF</a>
    <a id="on_f1g1" href="#" class="ui-btn ui-btn-inline">ON</a>

  </div>

  <div data-role="footer">
    <h1>Giridhar Bhujanga</h1>
  </div>
  
  <div data-role="footer">
    <h3>giridharmb@gmail.com</h3>
  </div>
</div> 

<script>
$(document).ready(function(){

  $.ajax({
    type: "GET"
    ,url: "http://www.my-server.in/mypath/arduino/get.php?object=F1L1"
    ,success:function(d) {
      console.info("--success--");
      console.info(d[2]);
      if(d[2] == "1") {
        $("#on_f1l1").addClass('ui-btn-b');
        $("#off_f1l1").removeClass('ui-btn-b');
      }
      if(d[2] == "0") {
        $("#off_f1l1").addClass('ui-btn-b');
        $("#on_f1l1").removeClass('ui-btn-b');
      }
    }
    ,error: function(e) {
      console.info("--error--");
      console.info(e[2]);
    }
  });

  $.ajax({
    type: "GET"
    ,url: "http://www.my-server.in/mypath/arduino/get.php?object=F1G1"
    ,success:function(d) {
      console.info("--success--");
      console.info(d[2]);
      if(d[2] == "1") {
        $("#on_f1g1").addClass('ui-btn-b');
        $("#off_f1g1").removeClass('ui-btn-b');
      }
      if(d[2] == "0") {
        $("#off_f1g1").addClass('ui-btn-b');
        $("#on_f1g1").removeClass('ui-btn-b');
      }
    }
    ,error: function(e) {
      console.info("--error--");
      console.info(e[2]);
    }
  });

  $("#off_f1l1").click(function(){
    $("#off_f1l1").addClass('ui-btn-b');
    $("#on_f1l1").removeClass('ui-btn-b');
    $.ajax({
    type: "GET"
    ,url: "http://www.my-server.in/mypath/mysql.php?variable=F1L1&value=0"
    ,success:function(d) {
      console.info("--success--");
      console.info(d);
    }
    ,error:function(e) {
      console.info("--error--");
      console.info(e);
    }
    });
  });

  $("#on_f1l1").click(function(){
    $("#on_f1l1").addClass('ui-btn-b');
    $("#off_f1l1").removeClass('ui-btn-b');
    $.ajax({
    type: "GET"
    ,url: "http://www.my-server.in/mypath/mysql.php?variable=F1L1&value=1"
    ,success:function(d) {
      console.info("--success--");
      console.info(d);
    }
    ,error:function(e) {
      console.info("--error--");
      console.info(e);
    }
    });
  });

  $("#off_f1g1").click(function(){
    $("#off_f1g1").addClass('ui-btn-b');
    $("#on_f1g1").removeClass('ui-btn-b');
    $.ajax({
    type: "GET"
    ,url: "http://www.my-server.in/mypath/mysql.php?variable=F1G1&value=0"
    ,success:function(d) {
      console.info("--success--");
      console.info(d);
    }
    ,error:function(e) {
      console.info("--error--");
      console.info(e);
    }
    });
  });

  $("#on_f1g1").click(function(){
    $("#on_f1g1").addClass('ui-btn-b');
    $("#off_f1g1").removeClass('ui-btn-b');
    $.ajax({
    type: "GET"
    ,url: "http://www.my-server.in/mypath/mysql.php?variable=F1G1&value=1"
    ,success:function(d) {
      console.info("--success--");
      console.info(d);
    }
    ,error:function(e) {
      console.info("--error--");
      console.info(e);
    }
    });
  });
});
</script>

File:mysql.php

<?php

    $varname = $_GET["variable"];
    $value = $_GET["value"];

    $servername = "localhost";
    $username = "dbuser";
    $password = "dbpassword";
    $dbname = "myvardb";

    // Create connection
    mysql_connect($servername, $username, $password ) or die(mysql_error());

    mysql_select_db($dbname) or die(mysql_error());


    $my_query = "INSERT INTO keyvalue (varname,varvalue) VALUES('" . $varname . "','" . $value . "') ON DUPLICATE KEY UPDATE varvalue='" . $value . "'";

    $retval = mysql_query( $my_query );

    if(! $retval )
    {
      die('Could not enter data: ' . mysql_error());
    }

    echo "SUCCESS";

    mysql_close($conn);
    
?> 

MySQL Dump of the DB/Table on the WWW server

-- MySQL dump 10.13  Distrib 5.6.19, for osx10.7 (i386)
--
-- Host: www.my-server.in    Database: myvardb
-- ------------------------------------------------------
-- Server version   5.5.32-cll-lve

/*!40101 SET @OLD_CHARACTER_SET_CLIENT=@@CHARACTER_SET_CLIENT */;
/*!40101 SET @OLD_CHARACTER_SET_RESULTS=@@CHARACTER_SET_RESULTS */;
/*!40101 SET @OLD_COLLATION_CONNECTION=@@COLLATION_CONNECTION */;
/*!40101 SET NAMES utf8 */;
/*!40103 SET @OLD_TIME_ZONE=@@TIME_ZONE */;
/*!40103 SET TIME_ZONE='+00:00' */;
/*!40014 SET @OLD_UNIQUE_CHECKS=@@UNIQUE_CHECKS, UNIQUE_CHECKS=0 */;
/*!40014 SET @OLD_FOREIGN_KEY_CHECKS=@@FOREIGN_KEY_CHECKS, FOREIGN_KEY_CHECKS=0 */;
/*!40101 SET @OLD_SQL_MODE=@@SQL_MODE, SQL_MODE='NO_AUTO_VALUE_ON_ZERO' */;
/*!40111 SET @OLD_SQL_NOTES=@@SQL_NOTES, SQL_NOTES=0 */;

--
-- Table structure for table `keyvalue`
--

DROP TABLE IF EXISTS `keyvalue`;
/*!40101 SET @saved_cs_client     = @@character_set_client */;
/*!40101 SET character_set_client = utf8 */;
CREATE TABLE `keyvalue` (
  `varname` varchar(256) NOT NULL,
  `varvalue` varchar(256) NOT NULL,
  PRIMARY KEY (`varname`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
/*!40101 SET character_set_client = @saved_cs_client */;

--
-- Dumping data for table `keyvalue`
--

LOCK TABLES `keyvalue` WRITE;
/*!40000 ALTER TABLE `keyvalue` DISABLE KEYS */;
INSERT INTO `keyvalue` VALUES ('F1G1','0'),('F1L1','0');
/*!40000 ALTER TABLE `keyvalue` ENABLE KEYS */;
UNLOCK TABLES;
/*!40103 SET TIME_ZONE=@OLD_TIME_ZONE */;

/*!40101 SET SQL_MODE=@OLD_SQL_MODE */;
/*!40014 SET FOREIGN_KEY_CHECKS=@OLD_FOREIGN_KEY_CHECKS */;
/*!40014 SET UNIQUE_CHECKS=@OLD_UNIQUE_CHECKS */;
/*!40101 SET CHARACTER_SET_CLIENT=@OLD_CHARACTER_SET_CLIENT */;
/*!40101 SET CHARACTER_SET_RESULTS=@OLD_CHARACTER_SET_RESULTS */;
/*!40101 SET COLLATION_CONNECTION=@OLD_COLLATION_CONNECTION */;
/*!40111 SET SQL_NOTES=@OLD_SQL_NOTES */;

-- Dump completed on 2015-02-12  9:54:38

Arduino Code

#include <Arduino.h>
#include <SoftwareSerial.h>
#include <WiFly.h>
#include "HTTPClient.h"
#include "Timer.h"

#define SSID      "WiFiNetworkName"
#define KEY       "WiFiPassword"
// WIFLY_AUTH_OPEN / WIFLY_AUTH_WPA1 / WIFLY_AUTH_WPA1_2 / WIFLY_AUTH_WPA2_PSK
#define AUTH      WIFLY_AUTH_WPA2_PSK

#define HTTP_GET_URL "http://www.my-server.in/giridharmb/arduino/get.php?object=F1L1"
#define HTTP_POST_URL "http://httpbin.org/post"
#define HTTP_POST_DATA "Hello world!"

// Pins' connection
// Arduino       WiFly
//  2    <---->    TX
//  3    <---->    RX
SoftwareSerial uart(2, 3);
WiFly wifly(uart);
HTTPClient http;
char get;

Timer t;

String response = "";
bool start_appending = false;

int RelayPin3 = 5; 

void setup() {
  pinMode(RelayPin3, OUTPUT);
  response = "";
  Serial.begin(9600);    
  Serial.println("------- WIFLY HTTP --------");
  
  
  uart.begin(9600);         // WiFly UART Baud Rate: 9600
  // Wait WiFly to init
//  delay(3000);

  // check if WiFly is associated with AP(SSID)
  if (!wifly.isAssociated(SSID)) {
    while (!wifly.join(SSID, KEY, AUTH)) {
      Serial.println("Failed to join " SSID);
      Serial.println("Wait 0.1 second and try again...");
      delay(100);
    }
    
    wifly.save();    // save configuration, 
  }


  Serial.println("\r\nTry to get url - " HTTP_GET_URL);
  Serial.println("------------------------------");
  while (http.get(HTTP_GET_URL, 10000) < 0) {
  }
  while (wifly.receive((uint8_t *)&get, 1, 1000) == 1) {
    Serial.print(get);
    if(get == '{')
    {
      start_appending = true;
    }
    if(get == '}')
    {
      start_appending = false;
      response = response + get;
    }
    if(start_appending == true)
    {
      response = response + get;
    }
  }
  
  Serial.println("\n---response---\n");
  Serial.println(response);
  Serial.println("\n--\n");
  
  if(response[1] == '1')
  {
    digitalWrite(RelayPin3,HIGH);
  }
  
  if(response[1] == '0')
  {
    digitalWrite(RelayPin3,LOW);
  }
  
  
  /*
  Serial.println("\r\n\r\nTry to post data to url - " HTTP_POST_URL);
  Serial.println("-------------------------------");
  while (http.post(HTTP_POST_URL, HTTP_POST_DATA, 10000) < 0) {
  }
  while (wifly.receive((uint8_t *)&get, 1, 1000) == 1) {
    Serial.print(get);
  }
  */
   
  if (wifly.commandMode()) {
    Serial.println("\r\n\r\nEnter command mode. Send \"exit\"(with \\r) to exit command mode");
  }
  
  t.every(15000, check_status);
   
    
}

void check_status()
{
  response = "";
   Serial.println("\r\nTry to get url - " HTTP_GET_URL);
  Serial.println("------------------------------");
  while (http.get(HTTP_GET_URL, 10000) < 0) {
  }
  while (wifly.receive((uint8_t *)&get, 1, 1000) == 1) {
    Serial.print(get);
    if(get == '{')
    {
      start_appending = true;
    }
    if(get == '}')
    {
      start_appending = false;
      response = response + get;
    }
    if(start_appending == true)
    {
      response = response + get;
    }
  }
  
  Serial.println("\n---response---\n");
  Serial.println(response);
  Serial.println("\n--\n");
  
  if(response[1] == '1')
  {
    digitalWrite(RelayPin3,HIGH);
  }
  
  if(response[1] == '0')
  {
    digitalWrite(RelayPin3,LOW);
  }
 
}

void loop() {
  t.update();
  int c;
  while (wifly.available()) {
    c = wifly.read();
    if (c > 0) {
      Serial.write((char)c);
    }
  }
  
  while (Serial.available()) {
    c = Serial.read();
    if (c >= 0) {
      wifly.write((char)c);
    }
  }
}

GITHUB Repo : iOS XCode Project

https://github.com/giridharmb/HomeController-iPhoneArduino.git