Posts By: jonny

Datanode denied communication with namenode

In a Cloudera cluster we had a few nodes reporting the following error message in the logs under hadoop-cmf-hdfs-DATANODE-whatever.com.log.out The error included: Datanode denied communication with namenode because the host is not in the include-list: : DatanodeRegistration One solution could be to ‘Refresh Node List’ as shown in the screenshot below. Find it by clicking… Read more »

Set NTP Server on Windows Domain Controller

On the Windows Domain Controller I checked the current NTP settings with:

  I tried a few alternative commands and got errors such as “the rpc server is unavailable”. The command that worked was:

After issuing that command I could see the ntp server listed when re-running the command:

As shown below.

Windows Nagios Checks using NSClient

Munin Alternative Template

I wanted to add some checks to a Windows server and the methods seem to have changed a little over the years. At the moment I have amended the nsclient.ini file on the Windows box with:

Then from the Nagios box I can check CPU with a threshold as follows:

We can also… Read more »

Browsing Hadoop HDFS on the Command Line

Using the Hadoop command we can send fairly regular filesystem commands to navigate HDFS. On Cloudera I am sending commands to port 8020 of the active name node. If you don’t have the hadoop command available on your system you need to download hadoop, extract it and add the bin directory to your path. For… Read more »

Labeling Partitions with LABEL and PARTLABEL

So working on a system that had incorrect partitioning configured.  In /etc/fstab the partitions were configured with:

but the partitions did not have PARTLABELS set. The LABEL and PARTLABEL can be viewed with:

To set the label property use:

To set the partlabel on a GPT partition first install gdisk:

Then… Read more »

Cloudera Hue User Password Reset

I needed to access the Hue file browser service  and for my version of cloudera this was: cd /opt/cloudera/parcels/CDH-5.7.6-1.cdh5.7.6.p0.6/lib/hue Run the createuser command: ./build/env/bin/hue createsuperuser Username (leave blank to use ‘joebloggs’): Email address: joebloggs@whatever.tld Password: Password (again): Superuser created successfully.

Mkr Fox 1200 Beginner Tutorial

Download the latest version of the Arduino IDE as the version in the Ubuntu/Mint/Debian repositories is a little old: https://www.arduino.cc/en/Main/Software At the time of writing I am using version 1.8.5 Configure Arduino IDE First we need to import everything we need to operate the MKR Fox 1200 so first visit Tools > Board > Board… Read more »

Zookeeper install on CentOS 7

Good concise article here on devopscube for installing zookeeper cluster. I’ve added some of my own bits: Firewall Disable firewalld or allow ports 2181 3888 2888 Install Java

User Create a zookeeper user

Install Zookeeper Download ZooKeeper from the Apache:

Set the permissions:

Add the following:

Test the service:… Read more »

Yakuake not restoring window height & width

git branch

I found that Yakuake was re-opening with an 80% width and about 30% height but I prefer 100% wide and 80% high. So it looks like my settings were saving into the file:

But this was possibly not the file that was being read when Yakuake started. I added the

entries to the… Read more »

Tab Separated File Returns Wrong Number of Fields

Shell

So we had a CSV/TSV file with tabs separating each field but one of the fields was a description field that contained spaces. When parsing this TSV file some programs interpreted the spaces as field delimiters (TSV might be a bad idea!). Input file similar to:

The ‘Time Warner Cable’ field is all one… Read more »