An Open MPI Master & Servant Example

Building on the Getting started… post from last week I’ve knocked up a quick example showing one way to get your MPI processes to communicate with one another.
master_servant.c:

#include <stdio.h>
#include <mpi.h>
#include <unistd.h>
int main(int argc, char *argv[]) {
int numprocs, rank, namelen;
char processor_name[MPI_MAX_PROCESSOR_NAME];
MPI_Init(&argc, &argv);
MPI_Comm_size(MPI_COMM_WORLD, &numprocs);
MPI_Comm_rank(MPI_COMM_WORLD, &rank);
MPI_Get_processor_name(processor_name, &namelen);
if ( rank == 0 ) {
printf( "[%02d/%02d %s]: I am the master\n", rank, numprocs, processor_name );
// Tell the servants to do something
} else {
printf( "[%02d/%02d %s]: I am a servant\n", rank, numprocs, processor_name );
// Wait for something to do
}
MPI_Finalize();
}

Build this with mpicc master_servant.c -o master_servant and run it, you should get something like:

[00/08 mpinode01]: I am the master
[01/08 mpinode01]: I am a servant
[02/08 mpinode01]: I am a servant
[03/08 mpinode01]: I am a servant
[04/08 mpinode02]: I am a servant
[05/08 mpinode02]: I am a servant
[06/08 mpinode02]: I am a servant
[07/08 mpinode02]: I am a servant

Ok, this means that based on the rank returned by MPI_Comm_rank we can decide which instance of the program is going to act as the “master” and which instance(s) are going to act as “servants” – pretty neat!
Next example, we build on this by getting the program instances to communicate with one another. Borrowed from the example found here.
master_servant2.c:

#include <stdio.h>
#include <string.h>
#include <mpi.h>
#include <unistd.h>
int main(int argc, char *argv[]) {
char idstr[32], buff[128];
int numprocs, rank, namelen, i;
char processor_name[MPI_MAX_PROCESSOR_NAME];
MPI_Status stat;
MPI_Init(&argc, &argv);
MPI_Comm_size(MPI_COMM_WORLD, &numprocs);
MPI_Comm_rank(MPI_COMM_WORLD, &rank);
MPI_Get_processor_name(processor_name, &namelen);
// Based on example from https://wiki.inf.ed.ac.uk/pub/ANC/ComputationalResources/slides.pdf
if (rank == 0) {
// This is the rank-0 copy of the process
printf("We have %d processors\n", numprocs);
// Send each process a "Hello ... " string
for(i = 1; i < numprocs; i++) {
sprintf(buff, "Hello %d... ", i);
MPI_Send(buff, 128, MPI_CHAR, i, 0, MPI_COMM_WORLD);
}
// Go into a blocking-receive for each servant process
for(i = 1; i < numprocs; i++) {
MPI_Recv(buff, 128, MPI_CHAR, i, 0, MPI_COMM_WORLD, &stat);
printf("%s\n", buff);
}
} else {
// Go into a blocking-receive waiting
MPI_Recv(buff, 128, MPI_CHAR, 0, 0, MPI_COMM_WORLD, &stat);
// Append our identity onto the received string
sprintf(idstr, "Processor %d ", rank);
strcat(buff, idstr);
strcat(buff, "reporting!");
// Send the string back to the rank-0 process
MPI_Send(buff, 128, MPI_CHAR, 0, 0, MPI_COMM_WORLD);
}
MPI_Finalize();
}

Build this example with mpicc master_servant2.c -o master_servant2 and run it, you should get the following output:

We have 8 processors
Hello 1... Processor 1 reporting!
Hello 2... Processor 2 reporting!
Hello 3... Processor 3 reporting!
Hello 4... Processor 4 reporting!
Hello 5... Processor 5 reporting!
Hello 6... Processor 6 reporting!
Hello 7... Processor 7 reporting!
Hello 8... Processor 8 reporting!

Now you can use this master/servant technique to partition work across instances of your MPI-capable program.
Troubleshooting
If you get an error like this “[hostname][0,1,0][btl_tcp_endpoint.c:572:mca_btl_tcp_endpoint_complete_connect] connect() failed with errno=113” try shutting down iptables on the MPI nodes. It was a quick-fix for me, i am sure there is a ‘proper’ way to configure it though. Keep in mind its probably not a good idea to switch off iptables on a machine if its connected to the open internet, the machines I have used in this guide are all on an internal network.