How to use your existing VirtualBox instance with Vagrant

In my earlier post, we saw how to use vagrant to spin up virtual development environment using VirtualBox. In this post, we’ll learn how to create base box from existing VirtualBox Linux instance(CentOS/Fedora) with Vagrant.

Guest Instance Changes

You’ll need to make the following changes to the existing instance from VirtualBox GUI:

  • In VirtualBox GUI, right click on your instance & go to settings –> Network
  • In Adapter1, check enable & choose attached to: NAT
  • click on Advanced, port forwarding –> add & set the following values:
    • Name: ssh
    • Protocol: TCP
    • Host IP: leave it empty
    • Host Port: 2222
    • Guest IP: leave it empty
    • Guest Port: 22
  • Click Ok.

Guest OS Changes

  • Log on to the instance & you’ll need to make following changes in the Guest OS:
    • Add vagrant user; sudo useradd vagrant and set password : vagrant
    • As root user, add vagrant to sudoers list & save it
      • # visudo
        • Defaults:vagrant !requiretty
        • Defaults env_keep = “SSH_AUTH_SOCK”
        • vagrant ALL=NOPASSWD: ALL
    • Update vagrant insecure key: required for login to the box
      • $ su – vagrant
      • $ mkdir -p ~/.ssh
      • $ chmod 0700 ~/.ssh
      • $ wget –no-check-certificate https://raw.github.com/mitchellh/vagrant/master/keys/vagrant.pub  -O ~/.ssh/authorized_keys
      • $ chmod 0600 ~/.ssh/authorized_keys
      • $ chown -R vagrant ~/.ssh
    • Install & Configure OpenSSH Server
      • $ sudo yum install -y openssh-server
      • $ sudo chkconfig sshd on
      • $ sudo vi /etc/ssh/sshd_config & update the following & save the file:
        • Port 22
        • PubKeyAuthentication yes
        • AuthorizedKeysFile %h/.ssh/authorized_keys
        • PermitEmptyPassword no
        • PasswordAuthentication no
    • Restart the ssh-service:  $ service sshd restart
    • Install VirtualBox GuestAdditions
      • $ sudo yum install -y gcc make kernel-devel
      • In the VirtualBox GUI, go to Devices > Install Guest Additions… from the menu bar
      • cd /media/VBOXADDITIONS_XXXXX
      • sudo ./VBoxLinuxAdditions.run
      • sudo umount /media/VBOXADDITIONS_XXXXX
    • Clean up the box to minimise the size & shutdown the guest OS
      • $ sudo yum clean all
      • $ Shutdown -h now

Package Guest Instance

  • On your host instance create a work folder
    • $ mkdir -p ~/workspace/centos64
    • $ cd ~/workspace/centos64
  • vagrant package –base <Guest Instance Name>. On successful completion, it will create a package.box file in the folder

Testing

  • Run the following from the folder, where package.box is stored
    • $ vagrant box add <instance_name> ~/package.box
    • $ vagrant box init : will initialise the box
    • $ vagrant up : will start the instance
  • To login to the Guest from host :
    • $ vagrant ssh
Advertisements

Avvy-Gavvy Sandwich

Avvy-Gavvy Sandwich

As my page says devops & more, here’s the more part. I like to cook. Here’s a simple egg sandwich which will satisfy your taste buds & hunger(I’ll add pictures as soon as I prepare it next time).

I’m calling this sandwich Avvy-Gavvy( after the name of my daughter’s pet lego piece)

Prerequisites

2 Brown Bread loaves

Mayonnaise

Veeba Barbecue Sauce ( Available on Amazon.in & supermarkets)

Veeba Sweet Onion Sauce ( Available on Amazon.in & supermarkets√)

Veeba Chipotle Sauce ( Available on Amazon.in & supermarkets)

Evolution of Avvy-Gavvy

In my days as a bachelor, I used to visit a small food joint run by middle aged generous couple, their son created a sandwich called Rohit’s special. It was simply delicious and it became a permanent weekend morning’s breakfast fix for me. Even after years, I still remember it’s taste, I was determined to re-create it at home.

I saw these Veeba Chipotle Sauce in supermarket, I brought it thinking I’ll use it with some snack . One fine day, I was preparing a regular bread – omelette, I thought I’ll give it a try to make it similar to Rohit’s special.

Iteration 1

I applied a mix of Mayonnaise, tomato ketchup and Veeba Chipotle sauce on the bread loaves and added a sunny side up omelette b/w loaves. It was good but not close to Rohit’s special.

Iteration 2

This time, I prepared caramelised onions. After applying the above mix, I added a layer of caramelised onions on top of bread loaves & added sunny side up omelette. It was better still not close to Rohit’s special

Iteration 3

During my next visit to superstore, I brought Veeba Barbecue sauce and Veeba Sweet Onion Sauce. This time, I ditched tomato ketchup & applied mix of mayyonaise, barbecue sauce, chipotle sauce & a little dash of sweet onion sauce on both the loaves, added caramelised onions & sunny side up omelette & the result was amazing. In my opinion this time it was as good as Rohit’s special.

I’m a big fan of Barbecue sauce’s smokiness, mayonnaise & chipotle sauce gives hints of tanginess & spiciness. Sweet Onion sauce balances the rest & Caramelised Onions adds to the texture

So that was the story of Avvy-Gavvy Sandwich, in case you decide to make one for yourself ; enjoy it & let me know the feedback.

How to use VirtualBox with Vagrant

In the previous post, we saw how to use homebrew to install VirtualBox & Vagrant. In this post, we’ll try to use vagrant to spin up virtual development environment using VirtualBox.

  • Create a folder in your user home:

$ mkdir -p project/vagrant
$ cd ~/project/vagrant
$ pwd
/Users/xxxxx/project/vagrant

  • To add a vagrant box, type the following command:

$ vagrant box add ubuntu/trusty64
==> box: Loading metadata for box ‘ubuntu/trusty64’
box: URL: https://vagrantcloud.com/ubuntu/trusty64
==> box: Adding box ‘ubuntu/trusty64’ (v1.0.0) for provider: virtualbox
box: Downloading: https://vagrantcloud.com/hashicorp/boxes/trusty64/versions/1.0.0/providers/virtualbox.box
box: Progress: 5% (Rate: 100k/s, Estimated time remaining: 0:25:15)

Search here to find a box suitable for your need.

  • Once the box is successfully added, you can initialise the box:

$ vagrant init trusty64

It will generate a vagrantfile & .vagrant folder
$ ls -la
drwxr-xr-x 3 xxxxx staff 102 mmm 13 22:16 .vagrant
-rw-r–r–     1 xxxxx staff 3024 mmm 13 22:40 Vagrantfile

  • To fire up the box:

$ vagrant up
Bringing machine ‘default’ up with ‘virtualbox’ provider…
==> default: This machine used to live in /Users/juhfa/projects/chef but it’s now at /Users/juhfa/projects.
==> default: Depending on your current provider you may need to change the name of
==> default: the machine to run it as a different machine.
==> default: Clearing any previously set forwarded ports…
==> default: Clearing any previously set network interfaces…
==> default: Preparing network interfaces based on configuration…
default: Adapter 1: nat
==> default: Forwarding ports…
default: 22 (guest) => 2222 (host) (adapter 1)
==> default: Running ‘pre-boot’ VM customizations…
==> default: Booting VM…
==> default: Waiting for machine to boot. This may take a few minutes…

  • Once the machine is successfully started, you’ll be able to see it added in the VirtualBox GUI. By default, port 2222 of your local machine is used to connect to port 22 of the virtual machine
  • You can then ssh to the box using:

$vagrant ssh
Welcome to Ubuntu —–
———————–
vagrant@trusty64:~$

How to install Vagrant and VirtualBox on Mac

Vagrant

Vagrant is developed by Hashicorp, it is an useful tool for creating virtualised instances using Virtualbox, VMWare, AWS etc. With a simple vagrantfile, you can create uniform virtual development instances & you can also use vagrant instances to test Shell scripts, PowerShell scripts etc.  It is also very useful in testing your Configuration management cookbooks/modules.

For download & more details refer: https://www.vagrantup.com/

VirtualBox

Virtualbox is a free and awesome virtualisation solution by Oracle. You can spin up multiple instance on your Mac using it, it’s very useful when you want to spin up different Operating System(OS) instances on your Mac such as Ubuntu/Centos/Windows etc.

All you need is Operating System iso’s. Add the iso, set desired configuration and power up the instance. You will need to install the Operating System for first use, thereafter instance will be ready for use.

For download & more details refer: https://www.virtualbox.org

Installation of VirtualBox and Vagrant

Regular Method

  • Download the installer to your Mac using safari browser.
  • Double click on the installation package & follow the instructions to install VirtualBox and Vagrant

Homebrew

It’s said to be the missing package manager for MacOS, I can’t agree more. I found it really useful.

  • To install homebrew: Paste this command in your terminal/iterm
/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
  • Add homebrew to your .bashrc/.bash_profile : export $PATH=$PATH:/usr/local/bin
  • To confirm installation is successful, Open a new terminal and type: brew doctor

Now we’ll use home-brew to install VirtualBox & Vagrant.

Go to terminal and type:

  • brew cask install virtualbox
  • brew cask install vagrant
  • brew cask install vagrant-manager

You should be able to see these applications in Launchpad.

Useful Linux Commands

Directories & Files

  1. To create multiple directories in one command & give it required permission

mkdir -pm 755 dir1/dir2/dir3

This command will create dir1, then create dir2 inside dir 1, then create dir 3 inside dir2 with 755 permissions on all dirs.

  1. To find the oldest file

ls -lt

ls -ltr → In reverse order

This command will sort the files by their modification time, or the last time it was changed.

  1. To find the newest file

ls -lu

  1. To find when was inode changed i.e. permission modification etc

ls -c

This command will list the files, as per the last changes in inode.

  1. To list all subdirectories

ls -R

  1. To list all directories

ls -l | grep ‘^d’

Alias ld=’ls -l | grep ‘^d’’

  1. To sort by Size

ls -lS

  1. To list the file with file types

ls -lAF

  1. To list the ending characters of the file

ls -l | cat -vte

cat -e option marks the end of lines with a $

  1. To list the file name in qoutes

ls -Q

  1. To list all hidden files

ls -la

  1. To list all hidden files except . & ..

ls -lA

  1. Counting files by type

find ${*-.} -type f -print | xargs file | awk ‘{ $1=NULL; t[$0]++; } END { for (i in t) printf (“%d\t%s\n”, t[i], i); }’ | sort -nr

Find command and it’s usage:

  1. To find all files in the present directory

find . -print

  1. why xargs option is used

When we give command ls -ld `find . -print` , it might give error, when command line is too long

hence we can give xargs

find . -print | xargs ls -ld

  1. How to find a file older than X days

find .  -type f -mtime X  -print  (mtime: modified time)

  1. How to find file not accessed for X days

find . -type f -atime X -print

  1. How to use “OR” operator in find command

find . -atime +5 \ ( -name “*.o” -o -name “*.tmp” \ ) -print

  1. Time operators which can be used with “find” command

atime = access time (tells last access time)

mtime = access time (tells last modification time)

ctime = access time (tells last inode changes time)

  1. How to use find for exact  File time comparisons using find command

create files with exact times you would like to compare
touch -t 201109092100 file1

touch -t 201110102200 file 2

find . -newer file 1 \! -newer file 2 -print
This will list all the files created between these 2 dates.

-exec operator in “find” command
Often, when you find a file, you don’t just want to see its name; you want to do something, like grep for a text string. To do this, use the -exec operator. This allows you to specify a command that is executed upon each file that is found.
The -exec operator allows you to execute any command, including another find command. If you consider that for a moment, you realize that find needs some way to distinguish the command it’s executing from its own arguments. The obvious choice is to use the same end-of-command character as the shell (the semicolon). But since the shell uses the semicolon itself, it is necessary to escape the character with a backslash or quotes.
Therefore, every -exec operator ends with the characters \; There is one more special argument that find treats differently: {}. These two characters are used as the variable whose name is the file find found.
The difference between -exec and xargs is subtle. The first one will execute the program once per file, while xargs can handle several files with each process. However, xargs may have problems with filenames that contain embedded spaces

  1. how to delete strange file using -exec operator in “find”

find the inode number of the file using ls -il
find . -inum 163866 -exec rm {} \;

  1. how to search for all files in a folder with required word using “find” command

find . -name \*.log  -exec grep -n “root.conf” {} \;
grep can also search for the word “root.conf” but it’ll search in all files, not just only in *.log files

  1.  how to find all symbolic links in your home dir & prints the files to which your symbolic links point

find $HOME -type l -ls | awk ‘{print $NF}’

  1. To search file by size

to search file > 500Bytes  c=Size in bytes
find . -size 500c -print
to search files < 500 Bytes
find . -size -500c -print

  1. To search files by permission, user , group

find . -perm 755 -print
find . -user maq -print
find . -group maq  755 -print

  1. duplicating a directory tree using find command

find . -type d exec mkdir \usr\project\{} \;
using sed :
find . -type d -print | sed ‘s@^@/usr/project/@’ | xargs mkdir

find . -type d -print | sed ‘s@^@mkdir @’ | (cd /usr/project; sh)

  1. How to limit find to search only files at first level i.e. stop it from searching recusively

find . -maxdepth 1 -print

  1. How to limit find to search only files in one disk partition only

find /usr -size +10485760c -xdev -print

  1. fastest way to search files on linux

locate filename

  1. To search a particular word or string in all files in a directory & it’s subdirectory

egrep ‘string’ `find . -type f -print`
-bash-3.2$ egrep ‘string’ `find . -type f -print`

find . -type f -print -exec egrep ‘string’ \;
In case there’s only 1 file in the directory, where you are searching, the find will only print the string and not the file name, to resolve this
find . -type f -print -exec egrep ‘string’  /dev/null

  1. How to find the top 5 big files

find . -type f -exec ls -s {} \; | sort -n -r | head -5

  1. how to find all hidden files

find . -type f -name “.*”

  1. Using {} more than 1 time in find command

find . -name “*.txt” -exec cp {} {}.bkup \;

  1. Redirecting errors to /dev/null

find / -name “*.conf” 2>>/dev/null

  1. how to subsitute space with _ in file names

find . -type f -iname “*.txt″ -exec rename “s/ /_/g” {} \;
Linking, Renaming & copying files

To Create a symllink/softlink

ln -s filename linkname

  1. To create a hardlink

ln filename linkname

  1. Word anchors: To search a pattern at the start of line : ^

To search a pattern @ end of line: $

  1. To search line starting with string “abcd”

grep ‘\<abcd\>’ server.txt

  1. To find a string that is a separate word (enclosed by spaces), it is better use the −w

for example to search “/” fs in /etc/fstab

if you give grep / /etc/fstab .. it’ll show all file systems having /

if you give grep -w /  /etc/fstab.. it’ll show only /root file system

  1. To search for . or * in a file

grep -F ‘*’ abc.sh

  1. To search all directory starting with a,b,c or x,y,z

ls -ld [a-cx-z]*

  1. To search all directory except the directories starting with a,b,c or x,y,z

ls -ld [^a-cx-z]*

  1. To print lines containing a pattern

sed ‘/string/p’ textfile

  1. To print only the lines matching the pattern

sed -n ‘/string/p’ textfile

  1. To delete the line containing the pattern

sed -n ‘/string/d’ textfile

  1. Find and replace with Sed

sed ‘s/errors/awesome’ abc.log

Note: This will change only 1st occurence of “errors” in the line

  1. Find and replace with Sed globally

sed ‘s/errors/awesome/g’ abc.log

  1. To insert a string “>” at the beginning of each line of a file

sed ‘s/^/> /’ abc.log

  1. To insert a string “EOL” at the end of each line of a file

sed ‘s/$/EOL/g’ abc.log

  1. To run multiple replace commands

sed ‘s/^/>/g’ -e ‘s/$/EOL/g’ abc.log
Keep in mind that by default sed prints its results to the standard output, most likely your terminal window. If you want to save the output to a file, redirect it

sed option ‘some/expression’ file_to_process > sed_output_in_a_file
AWK

How to print/display the first line of a file?

head -1 file.txt

sed ‘2,$ d’ file.txt

How to print/display the last line of a file?

tail -1 file.txt

sed -n ‘$ p’ test

How to display n-th line of a file?

sed –n ‘<n> p’ file.txt

Disclaimer:

All the trademarked and copyrighted content in my blog belong to their original owners and are included for representation purposes only.

Short script to remove docker images

  1. Create a file: remdockerimages.sh on your docker host machine
  2. Copy the below content in remdockerimages.sh
  3. #!/bin/bash
    rm imagesInfo.txt
    sudo docker images | awk ‘{print $3}’ > imagesInfo.txt
    echo “$(tail -n +2 imagesInfo.txt)” > imagesInfo.txt
    while read line; do
    sudo docker rmi “${line}”
    done < imagesInfo.txt
  4. Save the file and run ./remdockerimages.sh imagesInfo.txt
  5. Enjoy 🙂

GITHUB: https://github.com/tryinggithubnow/smalltools/blob/master/remdockerimages.sh

Short Script to remove all stopped/exited docker containers

  1. Create a file: remdockerinstances.sh on your docker host machine
  2. Copy the below content in remdockerinstances.sh
  3. #!/bin/bash
    rm instanceinfo.txt
    sudo docker ps -a | awk ‘{print $1}’ > instanceinfo.txt
    echo “$(tail -n +2 instanceinfo.txt)” > instanceinfo.txt
    while read line; do
    sudo docker rm “${line}”
    done < instanceinfo.txt
  4. Save the file and run ./remdockerinstances.sh instanceinfo.txt
  5. Enjoy 🙂

Docker Goodies

Thanks to Docker 🙂

Self-Paced Training for Docker

https://training.docker.com/self-paced-training

Docker Blogs

http://blog.docker.com (http://blog.docker.com/)

Docker Documents

http://docs.docker.com (http://docs.docker.com/)

References

https://github.com/kencochrane/docker-guidebook/blob/master/docker-guidebook.rst

https://github.com/wsargent/docker-cheat-sheet

http://www.oracle.com/technetwork/middleware/weblogic/overview/weblogic-server-docker-containers-2491959.pdf

http://linuxconfig.org/docker-container-backup-and-recovery

http://developerblog.redhat.com/2014/05/15/practical-introduction-to-docker-containers/

http://www.openstack.org/containers?gclid=CIDah9XDoMgCFVcVjgodkBADJw

http://blog.flux7.com/blogs/docker/docker-commands

Disclaimer:

All the trademarked and copyrighted content in my blog belong to their original owners and are included for representation purposes only.

A fresh start

I’m starting this blog dedicated to DevOps and more. I’ll try to keep this site updated with the DevOps related activities that I do, posts, articles, documents, the issues which i face while working with various DevOps tools & their Solutions, suggestions etc.