Tag: puppet

puppet

Configuration Management in puppet

Configuration Management in puppet

Will see How Configuration management puppet works in this post.

Let us take a example to create user in complex environment with different Linux distribution. To create a user we have small different in command when we go with different distribution like Red Hat, Ubuntu, CentOS,etc.

We have two method to create user without puppet help.

  1. We can directly login to the servers and will create user when the number of server is less. But, in when the server number hits more 100, its very difficult to create user manually in all user.
  2. We can create script to manage user in all servers. But, for that we should have knowledge about scripting and command different and flags(-u, -U) for each distribution. Once the script created, we need a common server which has access to all the other Linux servers.

But, using puppet we can do any type of user/group management, Package installation, service start/stop/restart, etc. By using puppet built-in resources to achieve the same operation on different distribution without worry about the underlying Operating System and commands.

By using simple code will do the necessary configuration management like
user/group management, Package installation, service start/stop/restart,etc.

Example: To create user will write below code to perform the task over all the Linux machines.

# cat user.pp
 user { "lbcuser1" :
  ensure => "present",
 }

Same like above if you want to delete a user/ install package, etc. Solution is wring simple, robust, idempotent, extendable puppet code to the necessary configuration over remote servers.

same like that will see the code to install ntp package, which is used for network time and starting service.

# cat ntp.pp
  package { "ntp":
         ensure => "present",
  } 

service { "ntpd":
         ensure => "running",
  }

Like this will manage environment using puppet code. In other work managing environment using code will call as Iac(Infrastructure-as-Code).
This code will be applied over all the client machines to do the operation and will reduce the manual effort and time.

And its very essay to change the code for any modification on configuration management over all client machines.

Idempotency:
Puppet codes are idempotent by nature. Which means the results of the code remains same irrespective of the number of time we perform puppet run on nodes.puppet always ensure to keep the resources in desired state.
For example in user creation, it will check whether the user is already exist.
If the user already exist, will not perform the user creation and report us that the user already exist. Basically these checks are already in place of the puppet resources.
And if you have lines of codes to perform a action on remote machines, in such case, if any of your action already exist in any server, puppet simply will skip that action and proceed for further configuration.

These all are the good points to why we are using puppet in our environment for configuration management.

Thanks for your support and reading this post. Will post next lecture about puppet in next post.

Refernce: Puppet Docs

About Puppet

Will see About puppet in this post. Puppet is a open source configuration management tool. Which will help us to reduce our working time by automating most of the day-to-day and other tasks in IT environment.

puppet is declarative one(Puppet domain specific language).
Puppet take care of all our regular repetitive task along with application deployment. configuration changes,etc.
Puppet written in Ruby.
Puppet is scale-label, which can be used any physical/virtual environments.
Codes written in puppet are idempotent by naturally.
It easily create/update and maintain the OS configuration files using its own declarative methods.

We can do below things using puppet on our OS without any human intervention.

* Installing application on various machines
* Managing Firewall ports
* Modifying configuration files
* Managing services, etc.

We have N number of Resources and Classes to build easily a complex environment over VMWare, Any Cloud environments.

How Puppet Works?
  • We have Master and agent concept in Puppet environment.
  • Master should be Installed and configured on Linux machines only and there is no support for Windows machine. But Agent can be Linux or Windows machines.
  • We have two deployment models
    • Master-Agent deployment : Master and agent machines different machines. Master will manage the agent machines. Its used for Production environment
    • Standalone deployment: Master and agent both packages are installed on one server and its used for Dev/Test Environment.
  • Puppet Master are Linux based machine where we need to install and configure “puppetserver” package and this will be responsible to create and maintain puppet codes to manage agent machines.
  • Agent machines are different servers in our environment which we would like to manage using puppet.
  • “Puppet-agent” package should be installed on agent machines
  • Agent machines will check with Master every “1800 Seconds(30 Mins)” to know if anything to be updated on agent machine.
  • If anything needs to be updated, Agent will pull from Master machine through puppet codes and this will be called us “pull mechanism” and will do required updates which is mentioned in puppet codes.
  • And we have “Push and Pull” based deployment.
  • In Push based, master will push the configuration updates to their agent machines
  • In Pull based model, Agents will establish connection with master and will pull the updates from master in periodic interval.
Workflow:
About Puppet
  • Administrator Login on Puppet Master to create/ Update puppet codes and this machine is responsible for puppet code management and contains different configurations in environment.
  • We have multiple agents in environment and puppet-agent package installed on agent machines.
  • Communication between master and agent will be established through secured certificates.
  • Puppet master will allow agent machines through port 8140
  • We make sure port 8140 enabled on firewall
  • Communication between master and agent has three steps
  • Once communication established, Agents send data to Master and the data includes, Host name, IP Address and MAC Address. These are called as facts.
  • Master uses this facts and compile a list with configuration which needs to be applied on agent and this will be called as catalog.
  • Catalog contains data such as packages to be installed/services, etc. which needs to be updated on agent machines based on puppet codes which wrote.
  • Agent uses the catalog to apply required changes on the nodes
  • Once agent received catalog, it will do required changes and nodes will report to master that will say the configuration has been applied ans successfully completed.
  • Puppet provides compatibility to get these reports using third party tools.

Reference: Puppet official Docs

Installing Puppet in RHEL7/ Cent OS 7

Will see Installing Puppet in RHEL7/ Cent OS 7

Nowadays all the administration tasks are rapidly moving to automation and its reducing manpower, cost and time. It’s mandatory to learn any one of the scripting and anyone of automation tool for system administrators at this situation.  Automating tasks in server end is very critical and most important one.  There are lots of tools available for automation. Here we are going to see the puppet.

What is puppet?

It’s an open source tool and developed by puppet labs for Linux/Unix/Mac and Windows machines. Puppet is written in Ruby language and its mostly used for automating jobs in the server and automating the jobs from the server to clients.

Features:

To avoid duplication of work.

Powerful framework

Mature tool

System administrator tasks simplify.

To make speed and repeated changes automatically.

To maintain system consistency and integrity

Very easy to manage vm, cloud and physical machines.

Enabling EPEL repository and dependencies for installation:

System time should be set as accurately on server, which is going to act as master chef server. So, set the time in your server.

Make sure that both the master and agent server should resolev the hostnames. So, make an entry in /etc/hosts file.

once time fixed will enable EPEL repository by using below command.

RHEL7/ Cent OS 7:

[root@server init.d]# rpm -Uvh https://dl.fedoraproject.org/pub/epel/epel-releas e-latest-7.noarch.rpm
Retrieving https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
warning: /var/tmp/rpm-tmp.yXngEr: Header V3 RSA/SHA256 Signature, key ID 352c64e 5: NOKEY
Preparing... ################################# [100%]
Updating / installing...
 1:epel-release-7-10 ################################# [100%]

List the EPEL Repository Packages :

[root@server init.d]# yum --disablerepo=* --enablerepo=epel list

Now to have puppet ready setup, have to use apache with passanger. To get that, download and place the repo file in /etc/yum.repos.d/

[root@server init.d]# curl --fail -sSLo /etc/yum.repos.d/passenger.repo https://oss-binaries.phusionpassenger.com/yum/definitions/el-passenger.repo

To setup puppet master/node now we need to enable the puppet lab repository and dependencies and run it on both master and agent servers.

[root@server init.d]# rpm -ivh https://yum.puppetlabs.com/puppetlabs-release-el-7.noarch.rpm
Retrieving https://yum.puppetlabs.com/puppetlabs-release-el-7.noarch.rpm
Preparing... ################################# [100%]
 package puppetlabs-release-22.0-2.noarch is already installed

 

Installing Puppet:

Use the below command to install Puppet server. This will install init script (/etc/init.d/puppetmaster) to run a test-quality puppet master server.

[root@server ~]# yum install puppet-server

Installed:
 puppet-server.noarch 0:3.8.7-1.el7

Dependency Installed:
 facter.x86_64 1:2.4.6-1.el7 hiera.noarch 0:1.3.4-1.el7 libselinux-ruby.x86_64 0:2.5-6.el7 libyaml.x86_64 0:0.1.4-11.el7_0
 puppet.noarch 0:3.8.7-1.el7 ruby.x86_64 0:2.0.0.648-29.el7 ruby-augeas.x86_64 0:0.4.1-3.el7 ruby-irb.noarch 0:2.0.0.648-29.el7
 ruby-libs.x86_64 0:2.0.0.648-29.el7 ruby-shadow.x86_64 1:2.2.0-2.el7 rubygem-bigdecimal.x86_64 0:1.2.0-29.el7 rubygem-io-console.x86_64 0:0.4.2-29.el7
 rubygem-json.x86_64 0:1.7.7-29.el7 rubygem-psych.x86_64 0:2.0.0-29.el7 rubygem-rdoc.noarch 0:4.0.0-29.el7 rubygems.noarch 0:2.0.14.1-29.el7

Complete!

Now we need to configure the puppet master/agent architecture.  in our case this node going to work as master server. So, edit /etc/puppet/puppet.conf file and change the dns_alt_names.

[root@server ~]# vi /etc/puppet/puppet.conf
[main]
 dns_alt_names=server,server.lbcdomain.com
 certname=server.lbcdomain.com

Save and exit from the file.

Run the below command to create puppet master certificate:

[root@server ~]# puppet master --verbose --no-daemonize

Once received “Notice: Starting Puppet master version 3.8.7” press ctrl+c to kill the process.

Now will configure Web server:

Puppet comes with basic web server. In production basically will not use this web server. Before managing nodes by puppet master server, will configure web server first.

[root@server ~]# yum -y install httpd httpd-devel mod_ssl ruby-devel rubygems gcc gcc-c++ pygpgme curl

Now install passenger and apache module:

[root@server ~]# yum install -y mod_passenger

create parent, public and tmp  directories for the application. Copy the ext/rack/config.ru from the puppet source code into the parent directory and set ownership as puppet to the directory.

[root@server ~]# mkdir -p /usr/share/puppet/rack/puppetmasterd
[root@server ~]# mkdir /usr/share/puppet/rack/puppetmasterd/public /usr/share/puppet/rack/puppetmasterd/tmp
[root@server ~]# cp /usr/share/puppet/ext/rack/config.ru /usr/share/puppet/rack/puppetmasterd/
[root@server ~]# chown puppet:puppet /usr/share/puppet/rack/puppetmasterd/config.ru

Create configuration file to add virtual host for puppet.

[root@server ~]# vi /etc/httpd/conf.d/puppetmaster.conf

make the below entries in virtual host file.

[root@server ~]# vi /etc/httpd/conf.d/puppetmaster.conf
# you probably want to tune these settings
PassengerHighPerformance on
PassengerMaxPoolSize 12
PassengerPoolIdleTime 1500
# PassengerMaxRequests 1000
PassengerStatThrottleRate 120

Listen 8140

<VirtualHost *:8140>
        SSLEngine on
        SSLProtocol             ALL -SSLv2 -SSLv3
        SSLCipherSuite          EDH+CAMELLIA:EDH+aRSA:EECDH+aRSA+AESGCM:EECDH+aRSA+SHA384:EECDH+aRSA+SHA256:EECDH:+CAMELLIA256:+AES256:+CAMELLIA128:+AES128:+SSLv3:!aNULL:!eNULL:!LOW:!3DES:!MD5:!EXP:!PSK:!DSS:!RC4:!SEED:!IDEA:!ECDSA:kEDH:CAMELLIA256-SHA:AES256-SHA:CAMELLIA128-SHA:AES128-SHA
        SSLHonorCipherOrder     on

        SSLCertificateFile      /var/lib/puppet/ssl/certs/server.lbcdomain.com.pem
        SSLCertificateKeyFile   /var/lib/puppet/ssl/private_keys/server.lbcdomain.com.pem
        SSLCertificateChainFile /var/lib/puppet/ssl/ca/ca_crt.pem
        SSLCACertificateFile   /var/lib/puppet/ssl/ca/ca_crt.pem
        # If Apache complains about invalid signatures on the CRL, you can try disabling
        # CRL checking by commenting the next line, but this is not recommended.
        SSLCARevocationFile     /var/lib/puppet/ssl/ca/ca_crl.pem
        # Apache 2.4 introduces the SSLCARevocationCheck directive and sets it to none
        # which effectively disables CRL checking; if you are using Apache 2.4+ you must
        # specify 'SSLCARevocationCheck chain' to actually use the CRL.
        # SSLCARevocationCheck chain
        SSLVerifyClient optional
        SSLVerifyDepth  1
        # The `ExportCertData` option is needed for agent certificate expiration warnings
        SSLOptions +StdEnvVars +ExportCertData

        # This header needs to be set if using a loadbalancer or proxy
        RequestHeader unset X-Forwarded-For

        RequestHeader set X-SSL-Subject %{SSL_CLIENT_S_DN}e
        RequestHeader set X-Client-DN %{SSL_CLIENT_S_DN}e
        RequestHeader set X-Client-Verify %{SSL_CLIENT_VERIFY}e

         DocumentRoot /usr/share/puppet/rack/puppetmasterd/public
        RackBaseURI /
        <Directory /usr/share/puppet/rack/puppetmasterd/>
                Options None
                AllowOverride None
                Order allow,deny
                allow from all
        </Directory>
</VirtualHost>  

Now restart the apache service.

[root@server ~]# systemctl restart httpd

Enable apache and disable puppet to autostart while booting.

[root@server ~]# systemctl disable puppet.service
[root@server ~]# systemctl enable httpd.service

Now enable puppet listening port in firewall:

Port: 8140

[root@server ~]# firewall-cmd --zone=public --add-port=8140/tcp --permanent
[root@server ~]# firewall-cmd --reload

 

Now install puppet on Node machines:

before starting puppet agent installation have to configure yum repository same like master server.

Command to install puppet:

[root@client ~]# yum -y install puppet

Make master puppet server details in puppet configuration file like below.

[root@node ~]# vi /etc/puppet/puppet.conf

[agent]
server = server.lbcdomain.com

Now start puppet service in node machine and enable it to start automatically while booting in this run level.

[root@client ~]# systemctl start puppet
[root@client ~]# systemctl enable puppet

Now signin puppet agent node machine certificate in puppet master server:

Agent node will request certificate on first time they attempt to run.

Login and execute the below command in master puppet server.

[root@server ~]# puppet cert list

"node.lbcdomain.com" (SHA256) D4:88:EC:C5:0A:F7:5D:4E:32:C5:B3:61:E0:51:7B:0C:CD:B3:49:9E:68:0B:E7:5D:75:19:1D:0B:92:8A:E7:C1

Run the below command to sign the cert sign

[root@server ~]# puppet cert sign node.lbcdomain.com

Notice: Signed certificate request for node.lbcdomain.com
Notice: Removing file Puppet::SSL::CertificateRequest node.lbcdomain.com at '/var/lib/puppet/ssl/ca/requests/node.lbcdomain.com.pem'

Run below command in puppet agent node machine to verify the certificate.

[root@client ~]# puppet agent -t

Info: Caching certificate for node.lbcdomain.com
Info: Caching certificate_revocation_list for ca
Info: Caching certificate for node.lbcdomain.com
Info: Retrieving pluginfacts
Info: Retrieving plugin
Info: Caching catalog for node.lbcdomain.com
Info: Applying configuration version '1445401911'
Info: Creating state file /var/lib/puppet/state/state.yaml
Notice: Finished catalog run in 0.09 seconds

Finally we completed puppet master and agent configuration successfully.