Category: linux

Now we are going to see how to enable User/Group disk quota enabling in Linux/Unix in this post. Sometimes we might have low space in on local disk. To avoid this will allocate disk size to Users/Groups by enabling and configuring quota in /home directory.

As a first step we should enable quota in Filesystem.

by editing and adding usrquota and grpquota in home directory entry at /etc/fstab file will enable quota.

[root@server ~]# vi /etc/fstab

# /etc/fstab
# Created by anaconda on Fri Nov 24 17:31:25 2017
#
# Accessible filesystems, by reference, are maintained under '/dev/disk'
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
#
/dev/mapper/rhel-root / xfs defaults 0 0
UUID=2f2c635e-e5fb-4c81-823a-855a334ca04c /boot xfs defaults 0 0
/dev/mapper/rhel-swap swap swap defaults 0 0
/dev/rootvg/lv_tmp1 /home xfs defaults,usrquota,grpquota 0 0

save and exit from the file.

Now remount the filesystem using below command.

[root@server ~]# mount -o remount /home

Then check whether the quota has been enabled or not in /home mountpoint.

[root@server ~]# mount | grep /home
 /dev/mapper/rootvg-lv_tmp1 on /home type ext4 (rw,relatime,seclabel,quota,usrquota,grpquota,data=ordered)

Creating database using below command

[root@server ~]# quotacheck -cugv /home

C:     Key to create new quota file

U:     User quota

G:     Group quota

V:     Verbose mode

And now turn on the quota in /home directory using below command

[root@server ~]# quotaon /home

Now will assign quota using edquota command to User/Group

Syntax to create quota on user:

#edquota -u <username>

Syntax to create quota on group:

#edquota -g <groupname>

Will see a example of creating quota on user called abu

[root@server ~]# edquota -u abu

Now the above command will open quota file like below

Disk quotas for user abu (uid 1001):
 Filesystem                 blocks soft hard inodes soft hard
 /dev/mapper/rootvg-lv_tmp1    0   5000 6000   0      0    0

Above data has two quota limits. One is based on blocks and another one based on inode.

For block usage:

Soft:    Soft limit will warn the user if the user exceeds the limit. But, user allowed to write data in home directory till reaching the hard limit. In above example, we have provided 5000KB(nearby 5MB)

hard:    Hard limit will not allow user to write data in home directory once reached hard limit. In above example, we have provided 6000KB(6MB) as hard limit.

Will login and try to create 8MB of file using dd command to check the quota on user.

[root@server ~]# su abu
[abu@server ~]$ dd if=/dev/zero of=bgfile bs=1M count=8
dm-3: warning, user block quota exceeded.
dm-3: write failed, user block limit reached.
dd: error writing ‘bgfile’: Disk quota exceeded
6+0 records in
5+0 records out
6127616 bytes (6.1 MB) copied, 0.00498719 s, 1.2 GB/s

Command to display report on user quota:

[root@server ~]# repquota -as
*** Report for user quotas on device /dev/mapper/rootvg-lv_tmp1
Block grace time: 7days; Inode grace time: 7days
 Space limits File limits
User used soft hard grace used soft hard grace
----------------------------------------------------------------------
root -- 20K 0K 0K 2 0 0
abu +- 6000K 5000K 6000K 6days 6 0 0

Will configure grace period for the user quota. Once grace period has been reached than the soft limit will be come hard limit.

Command to create grace period:

[root@server ~]# edquota -t

Grace period before enforcing soft limits for users:
Time units may be: days, hours, minutes, or seconds
 Filesystem Block grace period Inode grace period
 /dev/mapper/rootvg-lv_tmp1 7days 7days

Grace period also has two types which based on blocks and inodes.

Thanks for reading this post.

We are going to see Directory structure in Linux/ Unix and what was the use of those directories.

/: Root

Root is a parent directory for all the directories and files.

Each and every directories and files will comes under root only.

Only root user only will do any changes in this directory.

For root user /root is the home directory and for others home directory will comes under /home

/home

All the users home directory will be created under /home to store their files. Ex: /home/user

/boot

This directory contains boot loader information.

Boot loader file contains kernel and initramfs image details.

/bin

Contains all the executable binary files which are

commands which we are using in linux/unix.

/sbin

/sbin also contains binary files like same as /bin.

But, this commands are typically used by system administrator.

/etc

Contains configuration files of all the application/programs used in Linux/Unix.

and startup scripts also stored in this location.

/dev

This directory contains all the device files and drivers as well. Like CD Drive, HDD, USB, tty

/tmp

This directory is for temporary use only. All the temps files and directories stored here  which is created by user or system.

Files will be deleted after reboot of the system.

/opt 

Stands for optional.

This directory contains applications installed which all are separate vendor.

/var

Contains all the variable files and logs and  this can be grow in future based on the usage.

Ex:  /var/log/dmesg, /var/log/secure,etc…

/mnt

This will be used to mount devices temporary purpose.

/usr

This directory contains libraries, variables, binaries. /usr/bin directory contains binary files for user level programs and /usr/sbin contains binary files foe system administrator levels.

We are going to see  creating a new filesystem in Linux/Unix and especially in RHEL7.

As a first step we should know how many physical disks available in our machine and available free space in those disks.

For that we can use below commands. fdisk command will list all the physical disks with partitions and size of the disk.

[root@server ~]# fdisk -l

Disk /dev/sda: 21.5 GB, 21474836480 bytes, 41943040 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x00060f18

Device Boot Start End Blocks Id System
/dev/sda1 * 2048 1026047 512000 83 Linux
/dev/sda2 1026048 17803263 8388608 83 Linux
/dev/sda3 17803264 21997567 2097152 83 Linux
/dev/sda4 21997568 41943039 9972736 5 Extended
/dev/sda5 21999616 30388223 4194304 82 Linux swap / Solaris
/dev/sda6 30390272 41943039 5776384 8e Linux LVM

Disk /dev/sdb: 8589 MB, 8589934592 bytes, 16777216 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x1f51ec05

Device Boot Start End Blocks Id System
/dev/sdb1 2048 12584959 6291456 8e Linux LVM
/dev/sdb2 12584960 16777215 2096128 8e Linux LVM

Disk /dev/sdc: 8589 MB, 8589934592 bytes, 16777216 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

df command will list the free space in the mentioned disk

[root@server ~]# df -h /dev/sdc
Filesystem Size Used Avail Use% Mounted on
devtmpfs 1.4G 0 1.4G 0% /dev

We are going to use /dev/sdc disk, as its having free space and none FS created in this.

Use the below command to do changes and create filesystem in this disk

[root@server ~]# fdisk /dev/sdc
Welcome to fdisk (util-linux 2.23.2).

Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.

Device does not contain a recognized partition table
Building a new DOS disklabel with disk identifier 0x52240e5b.

Command (m for help):

Once you executed fdisk command with disk name(/dev/sdc), you will get in to the fdisk tool to do changes and it will show like above. type ‘m’ and press enter to get to know the keys which will be helpful in this tool

[root@server ~]# fdisk /dev/sdc
Welcome to fdisk (util-linux 2.23.2).

Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.

Device does not contain a recognized partition table
Building a new DOS disklabel with disk identifier 0x52240e5b.

Command (m for help): m
Command action
 a toggle a bootable flag
 b edit bsd disklabel
 c toggle the dos compatibility flag
 d delete a partition
 g create a new empty GPT partition table
 G create an IRIX (SGI) partition table
 l list known partition types
 m print this menu
 n add a new partition
 o create a new empty DOS partition table
 p print the partition table
 q quit without saving changes
 s create a new empty Sun disklabel
 t change a partition's system id
 u change display/entry units
 v verify the partition table
 w write table to disk and exit
 x extra functionality (experts only)

Now we are going to create a filesystem/partition. So for that type  n  to create new filesystem. Once pressed enter you will get prompt to select the partition type. Select Primary.

Than provide the partition number or else just press enter to take default value.

First Sector:  Provide the sector value, from there only filesystem/partition will start.

Last Sector:  Finally we have to provide the last sector value where the filesystem/partition will get end or in number value with G(GB),M(MB),K(KB). For best practise we can use the number value with + like this +4G and than press enter to complete it.

Command (m for help): n
Partition type:
 p primary (0 primary, 0 extended, 4 free)
 e extended
Select (default p): p
Partition number (1-4, default 1):
First sector (2048-16777215, default 2048):
Using default value 2048
Last sector, +sectors or +size{K,M,G} (2048-16777215, default 16777215): +4G
Partition 1 of type Linux and of size 4 GiB is set

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.

and now type  w  and press enter to save the changes and quit from the fdisk.

We created partition in physical disk. Use partprobe command to affect the disk changes without restarting the machine in this session.(Note: for hardware changes system restart is must. So here specially we are using this command to avoid machine reboot)

[root@server ~]# partprobe

[root@server ~]# fdisk -l /dev/sdc

Disk /dev/sdc: 8589 MB, 8589934592 bytes, 16777216 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x61352143

Device Boot Start End Blocks Id System
/dev/sdc1 2048 8390655 4194304 83 Linux

We have to format the partition using anyone of the filesystem type(XFS, EXT4,ETX3…).

Note: if you are going to use this partition in LVM, no need to follow the further steps.

As we are using RHEL7, going to use default one which XFS to format.

Command to format the partition using XFS filesystem.

[root@server ~]# mkfs.xfs /dev/sdc1
meta-data=/dev/sdc1 isize=512 agcount=4, agsize=262144 blks
 = sectsz=512 attr=2, projid32bit=1
 = crc=1 finobt=0, sparse=0
data = bsize=4096 blocks=1048576, imaxpct=25
 = sunit=0 swidth=0 blks
naming =version 2 bsize=4096 ascii-ci=0 ftype=1
log =internal log bsize=4096 blocks=2560, version=2
 = sectsz=512 sunit=0 blks, lazy-count=1
realtime =none extsz=4096 blocks=0, rtextents=0

After formating need to create a directory to mount this and the directory will act as mount point.

[root@server ~]# mkdir /app

Finally filesystem should be mounted under the created directory to make use it.

We ahev two type in mounting filesystem.

Temporary mount: Will mount the filesystem. But, after restarting mount will be lost. Again we need to mount it.

Permanent mount:  Need to make entry in /etc/fstab file and this mount will not lost even after restarting the machine as well.

Temporary mount:

[root@server ~]# mount /dev/sdc1 /app

Permanent mount:

Open the /etc/fstab file using vi editor and provide the below entry. Save and exit from the file using :wq.

[root@server ~]# vi /etc/fstab

/dev/sdc1 /app xfs defaults 0 0

Use mount command with grep to check the filesystem whether its listing or not.

[root@server ~]# mount | grep /app
/dev/sdc1 on /app type xfs (rw,relatime,seclabel,attr2,inode64,noquota)

Successfully we created a filesystem now.

We will see how to Increasing LVM in xfs filesystem in this post.

LVM is playing important role in Linux/Unix OS and it will help to increase/ reduce filesystem size in Linux/Unix OS.

We are going to see how to increase a filesystem in Linux/Unix especially in RHEL7 OS.

We can increase the filesystem in online and we no need to unmount the filesystem for this operation.

Steps to be followed in increasing filesystems:

First we need to check whether the free space available in the VG. if the required space available in the VG then will go head and extend simply using from step 4 to 6.

If there is no free space in VG, Follow the below steps.

1. Create a filesystem and change the filesystem type from “Linux” to “Linux LVM”(8e is the id/Hex code for Linux LVM filesystem type)
2. Create physical volume for that created filesystem.
3. Extend the Volume Group by adding the the created PV in this VG. now you will get free/unused space in your VG. Use “vgs” command to check that.
4. Now extend/increase the size of logical volume using “lvextend” command.
5. Than run “xfs_growfs” command to shrink the filesystem
6. Finally check the mount point size using “df -h <mountpoint>” and logical volume size using “lvs” command. Both size should be same after extending the filesystem as well.

In our case we are going to create a fileystem in existing physical disk unused space and than will create PV, vgextend and lvextend likewise.

Using below command will check the free spcae in our physical disk

[root@localhost ~]# df -h /dev/sdb
Filesystem Size Used Avail Use% Mounted on
devtmpfs 1.4G 0 1.4G 0% /dev

we have nearby 2GB space in existing disk.

Use below commands to create a filesystem:

[root@localhost ~]# fdisk /dev/sdb

use “n” key to create new partition and provide the required details to create it.

Use “t” key to change the filesystem type from “Linux” to “Linux LVM”

Below command will help us to check whether the filesystem created perfectly and change filesystem type. Here “/dev/sdb2” is theone which we created and changed filesystemtype.

Click here to know how to create a filesystem/partition in linux/unix

[root@localhost ~]# fdisk -l /dev/sdb

Disk /dev/sdb: 8589 MB, 8589934592 bytes, 16777216 sectors
 Units = sectors of 1 * 512 = 512 bytes
 Sector size (logical/physical): 512 bytes / 512 bytes
 I/O size (minimum/optimal): 512 bytes / 512 bytes
 Disk label type: dos
 Disk identifier: 0x1f51ec05

Device Boot Start End Blocks Id System
 /dev/sdb1 2048 12584959 6291456 8e Linux LVM
 /dev/sdb2 12584960 16777215 2096128 8e Linux LVM

Now create physical volume using below command:

[root@localhost ~]# pvcreate /dev/sdb2
 Physical volume "/dev/sdb2" successfully created.

As next step we need to extend the VG by adding the created PV in our VG.

Before that check the size and free size of our VG like below.

[root@localhost ~]# vgs
 VG #PV #LV #SN Attr VSize VFree
 newvg 2 2 0 wz--n- 11.50g 516.00m

Extend the newvg using below command:

And check the VG size and free space after extending the VG.

[root@localhost ~]# vgextend newvg /dev/sdb2
 Volume group "newvg" successfully extended
 [root@localhost ~]# vgs
 VG #PV #LV #SN Attr VSize VFree
 newvg 3 2 0 wz--n- 13.50g 2.50g

Now we have 2.5GB free space. SO, we are going to use this free spave to extend the LV.

Use lvs command to check avilabe LV’s and find the LV name which we are going to extend now.

[root@localhost ~]# lvs
 LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert
 lv1 newvg -wi-ao---- 4.00g
 lv2 newvg -wi-a----- 7.00g

Here we are going to extend lv1

Use below command to extend the LV1:

[root@localhost ~]# lvextend -L +2G /dev/newvg/lv1
 Size of logical volume newvg/lv1 changed from 4.00 GiB (1024 extents) to 6.00 GiB (1536 extents).
 Logical volume newvg/lv1 successfully resized.

And shrink the filesystem using below command:

[root@localhost ~]# xfs_growfs /dev/newvg/lv1

Finally check the mountpoint size and lv1 size using below command. Both should be same after extending the size.

[root@localhost ~]# df -h /lvolume1
 Filesystem Size Used Avail Use% Mounted on
 /dev/mapper/newvg-lv1 6.0G 33M 6.0G 1% /lvolume1

[root@localhost ~]# lvs /dev/newvg/lv1
 LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert
 lv1 newvg -wi-ao---- 6.00g 

size replicating in both check.

Successfully we extend the LV size.

In this post we are going to see Reducing LVM xfs format filesystem.

Reducing LV size on xfs fileystem is little different from reducing LV size on etx4/3/2 fileystem’s.

Follow the below steps to perform this:

1. As first step take data backup using xfsdump

2. Then unmount the fileysystem

3. Now shrink the logical volume to the required size using lvreduce command.

4. Format the filesystem using xfs format.

Note: While we reducing size over etx4/3/2 filesystems, will use resize2fs command to shrink instead of formating. So, we no need to take data backup while using etx4/3/2 filesystem for LVM.

5. Mount the filesystem again.

6. Now finally restore the data using xfsrestore command.

LVM details are below:

VG Name: newvg

LV’s: lv1, lv2. Going to use lv2 for this lab.

Use lvs command to check the size of existing LV’s

[root@localhost ~]# lvs
 LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert
 lv1 newvg -wi-ao---- 6.00g
 lv2 newvg -wi-a----- 7.00g

lv2 has been mounted under /lvolume2 mount point. Now take the backup of /lvolume2 under /tmp filesystem like below.

[root@localhost ~]#xfsdump -f /tmp/lv2.dump /lvolume2

Now unmount the filesystem using below command:

[root@localhost ~]# umount /lvolume2

Reduce or shrink the size to required size using lvreduce command:

In below example we are going to reduce the lv2 filesystem size from 7Gb to 6GB.

[root@localhost ~]# lvreduce -L -1G /dev/newvg/lv2
 WARNING: Reducing active logical volume to 6.00 GiB.
 THIS MAY DESTROY YOUR DATA (filesystem etc.)
 Do you really want to reduce newvg/lv2? [y/n]: y
 Size of logical volume newvg/lv2 changed from 7.00 GiB (1792 extents) to 6.00 GiB (1536 extents).
 Logical volume newvg/lv2 successfully resized.

Now format the filesystem using below command:

[root@localhost ~]#mkfs.xfs -f /dev/newvg/lv2

Mount the filesystem again into the /lvolume2

[root@localhost ~]#mount /dev/newvg/lv2 /lvolume2

Finally restore the backup data

[root@localhost ~]#xfsrestore -f /tmp/lv2.dump /lvolume2

Reference: yallalabs

We are going to see how to use sed command in Unix/Linux

SED – Stream Editor This is doing maximum operation over the file like insert, delete, search and replace the word or a character. mostly its used to find and replace operation in Linux/Unix environment.

Note: Once executed sed command, the output will be changed as per your requirement and original file remain the same.

Substituting or replacing string in a file using sed:

Mostly we are using sed to replace the string in a file. Use the below command to change a string. Here I am going to change platform as technology and I will use the below mentioned same file in all the examples.

[root@localhost ~]# cat > sed.txt
Linux is more secure and opensource operating system and Linux is low cast.
Multiuser is available in Linux and Linux is powerful OS.
Linux is the current and future OS in administration.

The command for substituting and replacing string in above file:

[root@localhost ~]# sed `s/Linux/Unix/` sed.txt
Unix is more secure and opensource operating system and Linux is low cast.
Multiuser is available in Unix and Linux is powerful OS.
Unix is the current and future OS in administration.

Command to change the nth occurrence of the string/pattern in a line:

Below command will change the nth string/pattern in line and we are going to change the second string.

[root@localhost ~]# sed 's/Linux/Unix/2' sed.txt
Linux is more secure and opensource operating system and Unix is low cast.
Multiuser is available in Linux and Unix is powerful OS.
Linux is the current and future OS in administration.

Command to  change the string/pattern which is matching to our string in a file:

Command will change all the string which matching to our given string/pattern in a file and we need to use g  option for this.

[root@localhost ~]# sed 's/Linux/Unix/g' sed.txt
Unix is more secure and opensource operating system and Unix is low cast.
Multiuser is available in Unix and Unix is powerful OS.
Unix is the current and future OS in administration.

Command to change nth occurrence of all the line in a file which matching to our given string/pattern:

This command will change all the nth occurrence in all the lines.

[root@localhost ~]# sed 's/Linux/Unix/2g' sed.txt
Linux is more secure and opensource operating system and Unix is low cast.
Multiuser is available in Linux and Unix is powerful OS.
Linux is the current and future OS in administration.

g is the key to change the occurrence globally in a file and here we are mentioning which occurrence have to be changed in a file which we mentioned as 2.

Command to set parenthesize first letter of all the characters in a line:

[root@localhost ~]# sed 's/\(\b[A-Z]\)/\(\1\)/g' sed.txt
(L)inux is more secure and opensource operating system and (L)inux is low cast.
(M)ultiuser is available in (L)inux and (L)inux is powerful (O)S.
(L)inux is the current and future (O)S in administration.

Here all the capital letters are taken as a first word and parenthesize has been set.

Command to replace the string/pattern in specific line:

[root@localhost ~]# sed '3 s/Linux/Unix/' sed.txt
Linux is more secure and opensource operating system and Linux is low cast.
Multiuser is available in Linux and Linux is powerful OS.
Unix is the current and future OS in administration.

Command to duplicate/print the modified line again:

Below command will print the modified line two times.

[root@localhost ~]# sed 's/Linux/Unix/p' sed.txt
Unix is more secure and opensource operating system and Linux is low cast.
Unix is more secure and opensource operating system and Linux is low cast.
Multiuser is available in Unix and Linux is powerful OS.
Multiuser is available in Unix and Linux is powerful OS.
Unix is the current and future OS in administration.
Unix is the current and future OS in administration.

Command to display/print the modified line alone:

in below example we are going to change the string in second line and will display the second line alone because its modified. We should use -n option along with /p tag in sed command.

[root@localhost ~]# sed -n '2 s/Linux/Unix/p' sed.txt
Multiuser is available in Unix and Linux is powerful OS.

Command to replace string/pattern in range of lines:

Command will replace the first occurrence in 2 and 3rd line.

[root@localhost ~]# sed '2,3 s/Linux/Unix/' sed.txt
Linux is more secure and opensource operating system and Linux is low cast.
Multiuser is available in Unix and Linux is powerful OS.
Unix is the current and future OS in administration.

Command to replace all the occurrence in rage of lines:

in below command string will be replaced in 2 and 3rd lines of all occurrence.

[root@localhost ~]# sed '2,$ s/Linux/Unix/g' sed.txt
Linux is more secure and opensource operating system and Linux is low cast.
Multiuser is available in Unix and Unix is powerful OS.
Unix is the current and future OS in administration.