Skip to content Skip to sidebar Skip to footer

How to Create a GPFS Filesystem on CentOS Linux

Hi everyone! This is my documentation about how to install and configure GPFS file system on top of CentOS Linux.
Just for your information, GPFS (General Parallel File System, brand name IBM Spectrum Scale) is a high-performance clustered file system software developed by IBM. It can be deployed in shared-disk or shared-nothing distributed parallel modes, or a combination of these. It is used by many of the world's largest commercial companies.

Here is the steps based on my experience on testing environment.

sudo yum install -y https://repo.ius.io/ius-release-el7.rpm
sudo yum update
sudo yum install nfs-utils ethtool rpcbind psmisc iputils ifupdown boost-regex kernel-devel cpp gcc gcc-c++ binutils ksh rpm-build m4
sudo yum install -y python36u python36u-libs python36u-devel python36u-pip python3





2. Extract downloaded zip > grant execution privilege > run the executable > press 1+enter on the License agreement, wait till finished.




3. Navigate to /usr/lpp/mmfs/5.1.1.3/gpfs_rpms/ then install the rpm package
rpm -ivh gpfs.base*.rpm gpfs.gpl*rpm gpfs.license*rpm gpfs.gskit*rpm gpfs.msg*rpm gpfs.compression*rpm gpfs.adv*rpm gpfs.crypto*rpm

4. Create portability layer
/usr/lpp/mmfs/bin/mmbuildgpl

5. Cluster creation. Follow the instruction from below link start from step 4

OR

Method 1. Add machine one by one using  
mmcrcluster -N /folder/nodelist or mmcrcluster –N node1:quorum/quorum-manager,node2

Method 2. Create Nodeslist (text file) then fill with node/cluster member

Message when cluster created successfully


Verify cluster status


6. Accept GPFS server license, the arguments can contain server or client and can have more than one node


7. Start GPFS on node1, mmstartup command should included on system startup

The start process could take some times to change from arbitrating to active


Do the same on the other node


8. Create NSD Configuration
The disk used by GPFS are called Network Shared Disk (NSD).
The file diskdef.txt below contains the NSD configuration used by GPFS.

One NSD disk has been defined, the name is mynsd1 and the device files of these disks are /dev/sdb (8GB). The disk will be used to store data and metadata.

%nsd:
device=/dev/sdb   <==== the disk used for GPFS
nsd=mynsd1   <==== the name
usage=data and metadata
 

If more than one disk needed, just repeat the step above and edit device=/dev/xxx.

Execute the process using command /usr/lpp/mmfs/bin/mmcrnsd -F diskdef.txt

Check the configuration


9. Create GPFS File System

/usr/lpp/mmfs/bin/mmcrfs fs_gpfs01 -F diskdef.txt -A yes -T /fs_gpfs01

The command above create a GPFS filesystem called fs_gpfs01 using the NSD defined in diskdef.txt file and will be mounted on /fs_gpfs01 mount point



10. Mount & Unmount

Mount the GPFS volume on all nodes :  /usr/lpp/mmfs/bin/mmmount /fs_gpfs01 -a
 

Useful Links
  • Install and configure GPFS 4.1 filesystem on Linux Centos 6.6 | Sysadmin continuous improvement (wordpress.com)
  • https://www.ibm.com/docs/en/spectrum-scale/5.1.1?topic=nodes-extracting-spectrum-scale-software-linux
  • https://www.ibm.com/docs/en/spectrum-scale/5.1.1?topic=bgplln-using-mmbuildgpl-command-build-gpfs-portability-layer-linux-nodes
  • https://www.ibm.com/docs/en/spectrum-scale/5.0.4?topic=reference-mmcrcluster-command
  • https://www.ibm.com/docs/en/spectrum-scale/5.0.4?topic=reference-mmaddnode-command#mmaddnode
  • https://www.ibm.com/docs/en/spectrum-scale/5.0.4?topic=reference-mmdelnode-command#mmdelnode
  • https://linuxize.com/post/how-to-install-python-3-on-centos-7/

Post a Comment for "How to Create a GPFS Filesystem on CentOS Linux"