Intel Xeon Phi Co-Processor Setup

Introduction

Just adding my notes here in case it helps anyone else who finds themselves in a similar situation to me.I have two machines each with an Intel Xeon Phi Co-Processor board. This (in my basic view of the world) is like an extra machine inside the host machine but contains 240 processors clocking in at 1Ghz each. The co-processor appears to use an Intel specific Linux system called k1om (I think it may be derived from SuSE) so Ubuntu and CentOS specific commands do not apply here.

For the purposes of this guide the following are my assumptions:

  • I will use the terms MIC, co-processor to refer to the co-processor board and the term host to refer to the system physically containing the co-processor board.
  • Our LDAP server is running at 192.168.161.75 with an LDAP base of hpc.myorg.com
  • hvan03 192.168.161.69 – this is the host machine containing the co-processor board
  • hvan03-mic0 192.168.161.169 – this is the co-processor board

The following websites/files are helpful for configuration:

* Intel MPSS Download Page
* Cluster Setup Guide
* MPSS User Guide
* MPSS LDAP User Guide

MPSS Installation

The MPSS rpms are available from The MPSS Downloads Section. I placed the RPM files into my personal Yum Repository. These MPSS RPMs are installed onto the host (hvan03) via a Puppet configuration.
The MPSS service can be managed with:

MIC Configuration

The mpss service should be stopped when making any config changes.
Upon initial installation of the MPSS software it is necessary to initialise the MIC configuration with:

This creates the necessary config files under /etc/mpss for the MIC which can then be modified.
The current MIC configuration can be viewed at any time with:

At this point if the MPSS service is started the co-processor will be accessible via SSH at the address specified in micctrl. e.g.
ssh root@172.31.1.1
The password for the root user on the MIC (hvan03-mic0) will be the same as the root password of the host (hvan03). The password hah is copied from the shadow file upon using –initdefaults.

Each MIC (co-processor board) is configured on the host (hvan03) at:

This config file includes details of the co-processor hostname, IP address, root directory location, image to be used
The filesystem for each MIC is under:

e.g. hvan03:/var/mpss/mic0/{etc,home,root,var}

Networking Configuration

It is necessary to create a bridge device on the host system (hvan03)
Create a bridge config file:

With the following contents:

Amend the primary network interface on the host (hvan03) for the bridge interface:

Add the following:

The mic0.conf will contain a ‘Network class=’ line with default settings as generated by the micctrl –initdefaults command earlier. Amend the network settings of the co-processor :

Change the ‘Network class’ line to the following:

Starting the MPSS service at this point should make the MIC (hvan03-mic0) accessible via SSH at the address 192.168.161.169 using the same root password as used on the host (hvan03). Hopefully from within the MIC you should be able to ping other machines on the same network (via the bridge).

LDAP Configuration

  • This configuration all takes place on the host (hvan03) rather than from within the MIC (hvan03-mic0)
  • I first downloaded the file mpss-3.4.3.k1om.tar file from the MPSS download site.
  • I extracted this tarball to /var/shared/k1om
  • This provides MPSS/k1om specific RPMs that can be installed within the MIC OS
  • Stop the mpss service: systemctl stop mpss
  • Add the location of the RPMs (on the host not the co-processor):

  • Set the LDAP details
  • Restart the MPSS service
  • Access the MIC via SSH using an LDAP username and password combination:
  • From within the MIC amend the /etc/ssh/sshd_config file to restrict logins and allow all users to connect from only from the host (but the admin can also ssh from another box):

NFS File Share Configuration

We have to mount the shares directly as there are no autofs packages for k1om.
Create a mount point for the shared NFS file share:

Amend /etc/fstab

Add the following:

Remount the fstab

Software Installation

I made use of the RPMs extracted to /var/shared/k1om from the mpss-3.4.3.k1om.tar file to install software that might be useful to our group:

 

Leave a Reply

  • (will not be published)

XHTML: You can use these tags: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code class="" title="" data-url=""> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong> <pre class="" title="" data-url=""> <span class="" title="" data-url="">