Unsolicited advice from the other side
For the young engineers out there interviewing, I have 4 general suggestions:
1- During the interview, do not drop industry-buzzwords that you don’t fully understand what they mean. For example, don’t say that you love chips manufactured by ARM. ARM does not make chips.
2- Come prepared with a portfolio with some of your work (e.g. source code, circuits, layouts, relevant blog posts) and leave a copy with the interviewers.This makes you look professional and organized.
3- In a technical engineering position no one is interested how cool the candidate was in college or how smart the candidate is for wanting to go to graduate school to learn more. In most companies there is a need for dedicated engineers to design, implement and maintain a project in its entire life-cycle. We simply have no time to train someone for a couple of months, so that they can jump-ship in six months to go to graduate school.
4- Go to interviews dressed business casual. I have noticed that the strongest candidates are usually the ones who don’t over-or-underdress.

Resume skill check
When we start the interview process, the first thing we do is re-read the skills section. If the candidate lists a particular skill, we will inevitably ask some very basic questions about it. For example, if C/C++ is listed as a skill, we generally ask the candidate to tell us the difference between C and C++. Unfortunately, only about 25% of the candidates know this.The simplest answer, as defined by the creator of C++ himself (Bjarne Stroustrup), is that C++ is C with classes.This would be the perfect time to talk about how C++ is an object oriented language and that objects allow the programer to secure (hide) data inside classes.The diligent engineer could also talk about how in C++, two functions can have the same name but different arguments (function overloading) and how variables in standard versions of C need to be declared at the start of each function. Similarly, if you do not understand the difference between Linux and Unix, do not put both on your resume.

Here are some of the commonly listed skills and some possible simple questions about them:
Matlab or Mathematica – Can you put 10 numbers in a 1D Matrix (orTable) and compute their average?
Linux – What is your Linux distribution? What command do you use to create and change a directory?
ASM – What does the command JMP do? What about MOV?
Python or Perl – Can you create a for loop that prints “hello world” 10 times?
HTML What does the and tag to?
Spice – Can you write a spice file with a circuit with a power supply and a single resistor?
Labview – Who makes Labview?

Finally, my recommendation is to avoid listing skills that you don’t fully understand what they mean. One of our candidates listed Xilinx as a software skill.This particular candidate not only did not know that Xilinx was a company, but he also could not remember the purpose of that particular piece of software (“I think it was to program some sort of chip.”).

Assessing the candidate interest in embedded systems
We are always interested in candidates with a passion for anything related to embedded systems, as these tend to be the strongest engineers. We often ask them where they buy their components, what electronics magazines they read and which websites they frequent. We also like to know what is their favorite prototyping system (e.g. arduino, raspi, beagleboard) and favorite programming language.We also like candidates to tell us about how they normally go about testing and implementing their prototypes. For example, how would they go about etching a PCB in their basement? While the answers to these questions are far from being deal-breakers, they do give us an insight about how passionate the candidate is about the field.
We also like our entry-level candidates to know a bit of whats out there. For example, what are the names of some companies that manufacture micro-controllers and the names of their products (e.g. Atmel manufactures ATmega328P which is used in the Arduino prototyping system).

The deal breakers
We have, what we like to call the “five deal breaker questions”. If candidate misses one of them, we cannot in good conscience hire them.
Question #1 – The most fundamental question regarding electrical circuit theory is how to measure current and voltage in a simple circuit. In the interview room we have a very simple circuit with a single resistor in a prototype board. After we power the circuit we give the candidate a digital multimeter and ask them to measure the voltage across, and the current through, the resistor; Voltage levels are measured by connecting the two probes across the resistor, while current is measured by opening the circuit and completing it with the multi-meter connected in series. Surprisingly about 10% of the candidates miss this question.

Question #2 – We need to know how confident the candidate is with basic circuit analysis. We draw on the board a schematic very similar to the one shown on Figure 1 and ask the candidate to determine the measured voltage across R2.

Figure 1 – Interview question: Determine the voltage across R2.

A confident engineer does not have to make any calculations.Two equal resistors in series with 10V across them, means that the voltage across just one of these resistors is 5V. The candidate can also determine this voltage by “hand”:

One quarter of our candidates, most with degrees in electrical engineering had trouble with this question.

Question #3 – We also ask very simple practical design questions. For example, when should you use a transistor instead of a relay in a design? There are many possible answers here, and we are just looking for the basic ones. For example, you can mention that the relay, since it is an electromechanical switch, is much slower than a transistor, but it can usually carry a lot more current. Relays in general are also much bigger than transistors.

Question #4 – It is fundamental that our new hires know the purpose of basic circuit concepts. Here are the three short-answer questions we always ask:

Q:What is a servo motor?

Q:What is a potentiometer?

Q:What is does it mean for a signal to have 30% duty cycle?

Question #5 – Finally, we always ask the candidate to explain what are the defining characteristics of an OP-AMP. Most candidates are able to tell us that the V+ and V- inputs have high impedance and they are often able to write out the (usually memorized) equations for OP- AMP circuits with a feedback loop. Unfortunately they can’t do much more than this, and the concept of the OP-AMP rail is often not understood.

We then ask the candidate to determine the output voltage of an ideal OP-AMP with a “comparator” configuration, exemplified in Figure 2.

Figure 2 – Interview question: Determine the voltage across R1.

Alternatively we also ask about the purpose of the voltage follower configuration, shown in Figure 3.

Figure 3 – Voltage follower op-amp configuration: If the voltage V1 is both at the V- and VM1, what is the purpose of this configuration?

Basic Embedded-C
We need our candidate to know how to program in C.There is no work around this one; the embedded systems engineer must know how to handle the most basic programming tasks.
Question #6 – The first programming question is always about distinguishing between the ‘=’ and ‘==’ operator (‘=’ is assignment, while ‘==’ is a comparator).
Question #7 – We then ask the candidate to write out the C-code that solves the commonly used fizzbuzz problem.The assignment is as follows:
“Write a program that prints the numbers from 1 to 100. But for multiples of three print “Fizz” instead of the number and for the multiples of five print “Buzz”. For numbers which are multiples of both three and five print “FizzBuzz”.”

While this is not a particularly hard, most of our candidates were unable to solve this assignment. Here is a possible solution:

int main() {
	for( int i=1; i<=100; i++)
	{
		if(i%3==0)
		  printf("Fizz");
		if(i%5==0)
		  printf("Buzz");
		if(i%3!=0 && i%5!=0)
		  printf("%d",i);

		printf("\n");
	}

	return 0;
}

We always ask something related to C-pointers. We understand that pointers can be a bit confusing to engineers, but the candidate must have at least a very general concept about what a pointer is.

Question #8 – Consider an integer k. How do you change its value through a pointer?

int a=6;
int *ptr=&a;
*ptr=10;

If the candidate shows confidence with pointers we normally ask slightly more advanced questions. For example,
Question #9 – What is the benefit of passing elements to a function by reference instead of by value? Can you write a small piece of code that demonstrates passing by reference?

int main(void) { 
    int a,b;
    a=10; b=20;
    swap_by_reference(&a, &b);
 
}
 
 void swap_by_reference(int *val_1, int *val_2)
 {
    int temp;
    temp=*val_1;
    *val_1 = *val_2;
    *val_2 = temp;
 }

Intermediate Embedded Systems topics
At this point we have a pretty good idea about the candidate and his abilities. If time allows we ask slightly more complex questions, allowing us to differentiate between the strongest candidates.

Question #10 – Consider the generic schematic on Figure 4. In this circuit, the solenoid L1 has been energized for a long time and suddenly we open the switch SW1. Can you foresee any problem? What can be done to fix it?

Figure 4 – Interview question: What might happens when we suddenly open SW1?

Figure 5 – Correct location and orientation of a flyback diode.

Question #11 – What is the purpose of an H-Bridge? Can you draw a generic H-bridge schematic?

Figure 6 – Generic implementation of an H-Bridge using npn-transistors.

Question #12 – Consider the very generic circuit on Figure 7. What happens when SW1 changes states?

Figure 7 – Interview question: What happens to D1 whenever SW1 is pressed?

Question #13 – What is an interrupt?

Question #14 – What is RS232?

Conclusion
Even with a relatively straight forward interview process, with a strong emphasis on the basics, we have had a difficult time recruiting skilled recent graduates. For those looking for a entry-level position, I strongly recommend to come prepared to the interviews. Spend a couple of days solving technical exercises straight from inexpensive books such as:

In addition keep reading and subscribing to magazines such as Elektor, Make, Circuit Cellar and, of course, Nuts And Volts.

I am running the following code in Arduino UNO. This particular CAN-BUS shield does not work with the latest Arduino board (Leonardo). Anyway, here is a simple Arduino program that continuously sends J1939 messages through the CAN network.

#include <Canbus.h>
#include <defaults.h>
#include <global.h>
#include <mcp2515.h>
#include <mcp2515_defs.h>

void setup()
{  
  Serial.begin(9600);
  
  //Initialise MCP2515 CAN controller at the specified speed
  if(Canbus.init(CANSPEED_250)) 
    Serial.println("CAN Init ok");
  else 
    Serial.println("Can't init CAN");

  delay(1000); 
}
 

void loop()
{
  static char counter=0;
  
  tCAN message;

  message.id = 0x00FF50FF;
  message.header.rtr = 0;
  message.header.length = 8;
  message.data[0] = 0x02;
  message.data[1] = 0x01;
  message.data[2] = 0xFF;
  message.data[3] = counter;
  message.data[4] = 0x00;
  message.data[5] = 0x00;
  message.data[6] = 0x00;
  message.data[7] = 0x00;						

  mcp2515_bit_modify(CANCTRL, (1<<REQOP2)|(1<<REQOP1)|(1<<REQOP0), 0);
  mcp2515_send_message_J1939(&message);
  counter++;  

  delay(1000);
}

The updated MCP2515 libraries can be found at http://www.nunoalves.com/source/canbus_nca26_09_2012.zip. To install this library and use it with your Arduino projects just drop it in the appropriate sub-directory (e.g. ~/Documents/Arduino/libraries in macosx). For reference here is the data sheet for the MCP2515 SPI CAN transceiver. I found all the information in there.

Finally… a big thank you to Fabian Greif and his initial efforts on the MCP2515 library.

A hex roman numeral is very much like the standard roman numeral, except with different values. In normal roman numerals, I = 1, V = 5, X = 10 and so on. In hex roman numerals, I = 1, V = 8, X = 16, L = 128, C = 256, D = 2048 and M = 4096. So for example:

VIIII = 8 + 1 + 1 + 1 + 1 = 12
IX = 16 – 1 = 15
XV = 16 + 8 = 24
XL = 128 – 16 = 112

The goal is to write a program in python that converts it in either direction. If given a decimal number, it should return the hex roman numeral version of the number and if given a hex roman numeral, it should return the decimal version of the number.

I started this by creating a program that performs a normal roman to arabic conversion. This wasn’t too hard, especially since python has a ton of neat features such as as dictionaries and solid string parsing methods. Since I am using unittest to test my code, I’ve named this file roman_numerals.py.

import sys, re

def roman_to_arabic(number):
	"""return the roman numeral string representation of integer number"""
	roman_dict={"I":1,"V":5,"X":10,"L":50,"C":100,"D":500,"M":1000}
	lst = [ roman_dict[i] for i in list(number) ]
	
	for n in xrange(len(lst)-1):
  		if (lst[n]<lst[n+1]): lst[n]=-lst[n]
  		
	return(sum(lst))

def arabic_to_roman(number):
	"""return the arabic numeral integer representation of roman string number"""
	units     = ("I","II","III","IV","V","VI","VII","VIII","IX","")
	tens      = ("X", "XX", "XXX", "XL", "L", "LX", "LXX", "LXXX", "XC", "")
	hundreds  = ("C", "CC", "CCC", "CD", "D", "DC", "DCC", "DCCC", "CM", "")
	thousands = ("M", "MM", "MMM", "MMMM","MMMMM","MMMMM","MMMMMM","MMMMMMM","MMMMMMMM","")

	#not quite sure how the romans dealt with very small or very large
	#numbers... also not quite sure how they worked with floating points
	assert(number<=7000)		
	assert(number>0)
		
	a=list(str(number))		#string-ify numbers
	b=a[-1::-1] 			#reverse order of list

	conversion=""
	
	if (len(b)>0): conversion=units[eval(b[0])-1]+conversion
	if (len(b)>1): conversion=tens[eval(b[1])-1]+conversion
	if (len(b)>2): conversion=hundreds[eval(b[2])-1]+conversion
	if (len(b)>3): conversion=thousands[eval(b[3])-1]+conversion
	return(conversion)		

if __name__== '__main__': 
	try:
		if (re.match("I|V|X|L|D|C|M", sys.argv[1])):
			print roman_to_arabic(sys.argv[1])
		else:
			print arabic_to_roman(eval(sys.argv[1]))
	except:
		print "Error: You either specified an: \n\t-invalid number \n\t-out of [1 to 7000] range \n\t-inexistent number"

The package unittest provides a great way to test your programs. I love it. You can pretty much run a another script and it will perform all the necessary assertions as it tests the proper package. Here is my unittest code, which I named test_roman_numerals.py.

import unittest, roman_numerals

class ProductTestCase(unittest.TestCase):
	def test_arabic_to_roman(self):
		self.failUnless("I"==roman_numerals.arabic_to_roman(1))
		self.failUnless("III"==roman_numerals.arabic_to_roman(3))
		self.failUnless("V"==roman_numerals.arabic_to_roman(5))
		self.failUnless("X"==roman_numerals.arabic_to_roman(10))
		self.failUnless("XI"==roman_numerals.arabic_to_roman(11))
		self.failUnless("VIII"==roman_numerals.arabic_to_roman(8))
		self.failUnless("IX"==roman_numerals.arabic_to_roman(9))
		self.failUnless("XV"==roman_numerals.arabic_to_roman(15))
		self.failUnless("XL"==roman_numerals.arabic_to_roman(40))
		self.failUnless("CXV"==roman_numerals.arabic_to_roman(115))
		self.failUnless("XLVI"==roman_numerals.arabic_to_roman(46))
		self.failUnless("MMXII"==roman_numerals.arabic_to_roman(2012))
		
	def test_roman_to_arabic(self):
		self.failUnless(1==roman_numerals.roman_to_arabic("I"))
		self.failUnless(3==roman_numerals.roman_to_arabic("III"))
		self.failUnless(5==roman_numerals.roman_to_arabic("V"))
		self.failUnless(10==roman_numerals.roman_to_arabic("X"))
		self.failUnless(11==roman_numerals.roman_to_arabic("XI"))
		self.failUnless(8==roman_numerals.roman_to_arabic("VIII"))
		self.failUnless(9==roman_numerals.roman_to_arabic("IX"))
		self.failUnless(15==roman_numerals.roman_to_arabic("XV"))
		self.failUnless(40==roman_numerals.roman_to_arabic("XL"))
		self.failUnless(115==roman_numerals.roman_to_arabic("CXV"))
		self.failUnless(46==roman_numerals.roman_to_arabic("XLVI"))
		self.failUnless(2012==roman_numerals.roman_to_arabic("MMXII"))

if __name__== '__main__': unittest.main()

Here are some screenshots of the program in action: first testing through the command line, and then testing it with unittest.

Taking this base code and making it compatible with base-16 numerals was trivial. All that I needed to do was make a minor modification to the dictionary roman_dict and adding extra elements to the lists units, tens, hundreds and thousands. Of course I had to perform a base conversion with the hex2dec function each time I wanted to access a position in the list.
Here is the code that converts base-16 roman numbers to arabic numbers. I saved this file as roman_numerals_base16.py.

import sys, re

def roman_to_hex_arabic(number):
	"""return the roman numeral string representation of integer number"""
	roman_dict={"I":1,"V":8,"X":16,"L":128,"C":256,"D":2048,"M":4096}
	lst = [ roman_dict[i] for i in list(number) ]
	
	for n in xrange(len(lst)-1):
  		if (lst[n]<lst[n+1]): lst[n]=-lst[n]
  		
	return(sum(lst))

def arabic_to_hex_roman(number):
	"""return the arabic numeral integer representation of roman string number"""
	units     = ("I", "II", "III", "IIII", "IIIII", "IIIIII", "IV", "V", "VI", "VII", "VIII", "VIIII", "VIIIII", "VIIIIII", "IX", "")
	tens      = ("X", "XX", "XXX", "XXXX", "XXXXX", "XXXXXX", "XL", "L", "LX", "LXX", "LXXX", "LXXXX", "LXXXXX", "LXXXXXX", "XC", "")
	hundreds  = ("C", "CC", "CCC", "CCCC", "CCCCC", "CCCCCC", "CD", "D", "DC", "DCC", "DCCC", "DCCCC", "DCCCCC", "DCCCCCC", "CM", "")
	thousands = ("M", "MM", "MMM", "MMMM","MMMMM","MMMMM","MMMMMM","MMMMMMM","MMMMMMMM","")

	#not quite sure how the romans dealt with very small or very large
	#numbers... also not quite sure how they worked with floating points
	assert(number<=7000)		
	assert(number>0)
		
	a=list(str(dec2hex(number)))	#string-ify numbers
	b=a[-1::-1] 					#reverse order of list

	conversion=""
	
	if (len(b)>0): conversion=units[hex2dec(b[0])-1]+conversion
	if (len(b)>1): conversion=tens[hex2dec(b[1])-1]+conversion
	if (len(b)>2): conversion=hundreds[hex2dec(b[2])-1]+conversion
	if (len(b)>3): conversion=thousands[hex2dec(b[3])-1]+conversion
	return(conversion)		

def dec2hex(n):
    """return the hexadecimal string representation of integer n"""
    return "%X" % n

def hex2dec(s):
    """return the integer value of a hexadecimal string s"""
    return int(s, 16)
        		

if __name__== '__main__': 
	try:
		if (re.match("I|V|X|L|D|C|M", sys.argv[1])):
			print roman_to_hex_arabic(sys.argv[1])
		else:
			print arabic_to_hex_roman(eval(sys.argv[1]))
	except:
		print "Error: You either specified an: \n\t-invalid number \n\t-out of [1 to 7000] range \n\t-inexistent number"

And here are my test cases, taken directly from the problem statement and saved as test_roman_numerals_base16.py.

import unittest, roman_numerals_base16

class ProductTestCase(unittest.TestCase):
	def test_arabic_to_hex_roman(self):
		self.failUnless("VIIII"==roman_numerals_base16.arabic_to_hex_roman(12))
		self.failUnless("IX"==roman_numerals_base16.arabic_to_hex_roman(15))
		self.failUnless("XV"==roman_numerals_base16.arabic_to_hex_roman(24))
		self.failUnless("XL"==roman_numerals_base16.arabic_to_hex_roman(112))		
		self.failUnless("XI"==roman_numerals_base16.arabic_to_hex_roman(17))		

	def test_roman_to_hex_arabic(self):
		self.failUnless(12==roman_numerals_base16.roman_to_hex_arabic("VIIII"))
		self.failUnless(15==roman_numerals_base16.roman_to_hex_arabic("IX"))
		self.failUnless(24==roman_numerals_base16.roman_to_hex_arabic("XV"))
		self.failUnless(112==roman_numerals_base16.roman_to_hex_arabic("XL"))
		self.failUnless(17==roman_numerals_base16.roman_to_hex_arabic("XI"))
		
if __name__== '__main__': unittest.main()

Finally, here is a screenshot of the program in action.

Lets say you have a directory tmp with a bunch of files you want to compress and password protect it. You can go the standard route using the zip utility. Just type the following:

There is a hack you can use to bypass the manually password submission… It involves using expect. This program is a really neat utility. It basically allows you to create interactive dialogues with your terminal programs, which makes task automation a walk in the park. You can check its manual pages here.

Step #1: Go into the mac terminal and find out where the expect utility is located. This is done using the whereais command. Type the following on your terminal window.

In my macosx version, the program expect is located at /usr/bin/expect.
Step #2: Create a file (e.g. protect_directory.sh) with the following lines. Make sure you modify the location of the expect program in the first line of the script.

#!/usr/bin/expect -f
eval spawn zip [lindex $argv 0].zip -R [lindex $argv 1]/* -e
expect "password:"
send "[lindex $argv 2]\r"
expect "password:"
send "[lindex $argv 2]\r"
interact

Step #3: Make sure your script is executable and run it.

Anyway, on the weekends I also love spending time with my son. Unfortunately, my son is 1.3 years old which means i can’t play with both at the same time. Time to optimize! I created a very simple system that repeatedly presses the same button over an over again. When the character is in an adequate location, it will repeatedly kill any enemies that are walking in his direction.

The point? I want to play the hard bosses and skip all the grinding. This simple contraption does the job. It is not pretty (all materials used were scavenged from here and there) but it works.

Here are two videos showing the machine in action…

And of course the Arduino source code… Everything was made, assembled and implemented in about 15 minutes. Automation FTW!

#include <Servo.h> 
 
Servo myservo; 
int potinput=0;
int delayvalue=0;

void setup() 
{ 
   myservo.attach(9); 
   pinMode(10, OUTPUT);
   digitalWrite(10,HIGH);
} 
 
void loop() 
{ 
   myservo.write(170);
   delay(delayvalue); 
   myservo.write(120);
   delay(delayvalue); 
   potinput=analogRead(0); 
   delayvalue = map(potinput, 0, 1022, 100, 2000);
}

Moving a duck with Python and curses

import curses, random
	
def draw_duck(top_pos,left_pos):
	duck=(
 		"    _      ",
 		"   =')_//  ",
 		"   (___/   ")
	screen.clear() 

	for iterator in duck:
		screen.addstr(top_pos, left_pos, iterator)
		top_pos+=1

def update_screen(top_pos,left_pos,worm_location,worms_eaten):
	draw_duck(top_pos,left_pos)
	if (top_pos+1==worm_location[0]) and (left_pos+1==worm_location[1]):
		worms_eaten+=1
		worm_location=[
			random.randrange(1, screen_dimensions[0]),
			random.randrange(1, screen_dimensions[1])
			]
	screen.addstr(worm_location[0], worm_location[1],"~")
	say_this="Score=%d duck=(%d,%d) worm=(%d,%d)" % (worms_eaten, top_pos , left_pos,  worm_location[0], worm_location[1])
	screen.addstr(21,0,say_this)
	return (worm_location,worms_eaten)

screen = curses.initscr() 
curses.noecho() 
curses.curs_set(0) 
screen.keypad(1)

worm_location=[
	random.randrange(1, screen_dimensions[0]),
	random.randrange(1, screen_dimensions[1])
	]
worms_eaten=0

screen.addstr("Use the cursor keys to move the duck!\n") 
top_pos  = 1
left_pos = 1 
while True: 
	event = screen.getch() 

	if event == ord("q"): break 
	elif event == curses.KEY_UP: 
		top_pos-=1
		if (top_pos<=0): top_pos=0
		if (top_pos>=20): top_pos=20
		worm_location, worms_eaten = update_screen(top_pos,left_pos,worm_location,worms_eaten)
		
	elif event == curses.KEY_DOWN: 
		top_pos+=1
		if (top_pos<=0): top_pos=0
		if (top_pos>=20): top_pos=20
		worm_location, worms_eaten = update_screen(top_pos,left_pos,worm_location,worms_eaten)
		
	elif event == curses.KEY_LEFT: 
		left_pos-=1
		if (left_pos<=0): left_pos=0
		if (left_pos>=20): left_pos=20
		worm_location, worms_eaten = update_screen(top_pos,left_pos,worm_location,worms_eaten)

	elif event == curses.KEY_RIGHT: 
		left_pos+=1
		if (left_pos<=0): left_pos=0
		if (left_pos>=20): left_pos=20
		worm_location, worms_eaten = update_screen(top_pos,left_pos,worm_location,worms_eaten)
     
	elif event == ord(" "): 
		screen.clear() 
		screen.addstr("Press 'q' to exit")
      		
curses.endwin()

Anyway, I still think that audio based interactive entertainment systems have potential, but my technical solution I chose was not the best. Most definitely I will revisit this idea soon with Android phones and tablets. Implementing this in android seems to be very easy, thanks to the IOIO connectors. For those interested, this connector is available for purchase through sparkfun. Regardless, here is a very simple code for my choose your own adventure. It is done in python and if you have a macosx it will read out the entries and choices using the built in speech synthesizer.

import os 

#reads the current entry
def read_entry(entry_number, filename):
	f=open(filename,'r')
	found=False	
	for line in f.readlines():
		line=line.strip('\n')
		if line == '<'+str(entry_number)+'>':
			found=True	
		if (line == '[choices]') or line == '[end]':
			found=False	
		if (found==True) and (line!='<'+str(entry_number)+'>'):
			print line
			saythis="say %s" % line
			os.system(saythis) 
			
#creates a dictionary with the choices at each entry
def read_choices(entry_number, filename):
	f=open(filename,'r')
	entry_found=False
	choice_section=False	
	items={}
	for line in f.readlines():
		line=line.strip('\n')
		if line == '<'+str(entry_number)+'>':
			entry_found=True	
		if (entry_found==True) and (choice_section==True) and (line!='[end choices]'):			
			#remove ">" from line 
			line=line.replace('>','')
			#remove "-" from line 
			line=line.replace('-','')
			
			line=line.split('<')
			x={line[0] : line[1]}
			items.update(x)
	
		if (entry_found==True) and (line=='[choices]'):
			choice_section=True		
		if (entry_found==True) and (line=='[end choices]'):			
			choice_section=False		
			entry_found=False	
		if (line=='[end]'):
			entry_found=False			
	return items

def ask_choices(choices):	
	saythis="What do you want to do?"
	print saythis
	saythis="say %s" % saythis
	os.system(saythis) 
	i=1
	for current_choice in list(choices):
		saythis='Choice number %d:' % i + '%s' %current_choice
		print saythis
		saythis="say %s" % saythis
		os.system(saythis) 
		i+=1
	saythis="What is your choice?"
	print saythis
	saythis="say %s" % saythis
	os.system(saythis) 
	user_choice=input('')
	key=list(choices)[user_choice-1]
	#the function return value is next story page
	return choices.get(key)
	
story_page=1
terminate=False
while terminate==False:		
	read_entry(story_page,'test_story.txt')
	choices=read_choices(story_page,'test_story.txt')
	#if there are no choices, then we reached the end
	if len(list(choices))==0:
		terminate=True 
	else:
		story_page=ask_choices(choices)

The story itself (hardwired on the previous file as test_story.txt) is pretty self-explanatory.

Running the python script...

Enjoy.

Here I outline the steps I took to extract all job postings from DICE. First of all you have to know how everything is stored in the database. Make any random search on the initial screen (e.g. embedded).

"Embedded" search on dice.com

This particular search generated the following very-long URL… so long that I had to include spaces:

Since this particular search detected 1689 job postings, we just have to change NUM_PER_PAGE=30 from 30 to 1689, in order to see every single job post on a single page. Save that file into your hard-disk in the HTML format. For completeness, here is the file with all 1689 postings I just downloaded. The following PERL script parses the contents of this file and looks for the associated URL for each job posting.

#neat function that helps catching scripting errors like undefined variables...
use strict;

#use the generated HTML as an input argument
my $input_file=$ARGV[0];

#parse through every single line in the file
open (MYFILE, "<$input_file") or die $!;
 while (<MYFILE>) {
 	chomp;
 	#each job posting starts with a well defined pattern...
	#search for <div><a href="/jobsearch/servlet/Jo
	if (/<div><a href=\"\/jobsearch\/servlet\/Jo/)
	{
		#...the following line:
		#print "$_\n";
		#...will print for example:
		#<div><a href="/jobsearch/servlet/JobSearch?op=302&amp;dockey=xml/7/0/70b58be12a872e8268939a525389b927@endecaindex&amp;source=19&amp;FREE_TEXT=embedded&amp;rating=99">ASIC/FPGA Verification Engineer</a></div>

		#get the job title
		my $job_title;
		my @tmp_data;

		@tmp_data=split(/">|<\/a><\/div>/,$_);		
		$job_title=$tmp_data[1];
		#...the following line:
		#print $job_title . "\n";
		#...will print for example:
		#ASIC/FPGA Verification Engineer


		@tmp_data=split(/">|div><a href="\//,$_);
		#...the following line:
		#print $tmp_data[1] . "\n";	
		#...will print for example:	#jobsearch/servlet/JobSearch?op=302&amp;dockey=xml/7/0/70b58be12a872e8268939a525389b927@endecaindex&amp;source=19&amp;FREE_TEXT=embedded&amp;rating=99

		#but that is not the correct URL... instead we want something like:	#http://seeker.dice.com/jobsearch/servlet/JobSearch?op=302&dockey=xml/b/4/b4ba4b9ed60baf2cf7a3397f336e451e@endecaindex&source=19&FREE_TEXT=embedded&rating=0

		#in essence, we must replace all &amp; with &
		$tmp_data[1] =~ s/\&amp;/\&/g;
		$tmp_data[1] =~ s/\/jobsearch\/servlet//g;
		print "http://seeker.dice.com/" . $tmp_data[1] . "\n";
	}
 }
 close (MYFILE);

Save the file (e.g. dice.pl) and execute it with the following command:

The next step is to download every single job posting onto a separate file. Since, I am a macsox user, I need to download a the contents of each of the URLs from the web via the OS X command line. This is easy accomplished with the following bash script:

FILENAME=embedded_url.txt

NUMBERLINES=`wc -l <  ${FILENAME}`
echo ${NUMBERLINES} > xxx.tmp
PNUMBERLINES=`perl -n -e '@splitline=split(/ /,$_); $splitline[1]=~s/ //g; print $splitline[0] ."\n"; ' xxx.tmp`
rm -f xxx.tmp
echo ${PNUMBERLINES}

RUN=0
until [ ${RUN} -eq ${PNUMBERLINES} ]
do
    RUN=$(( $RUN + 1 ))
    LCONTENTS=`sed -n $RUN'p' ${FILENAME}`
    #echo "line # ${RUN} with contents : ${LCONTENTS}"
    wget "${LCONTENTS}"
done

On the same directory as the output of the previous PERL script (e.g. embedded_url.txt), save this bash scrip (e.g. download_all_jobs.sh) and execute it with the following commands:

The compressed outcome of this last step is a file of 27 MBs.

Now that I have all this data, I need to extract the skills required for each advertised position. So, I placed all the compressed files in the sub-directory dice_jobs and ran the following script:

#!/bin/bash
JOBS_DIRECTORY=dice_jobs

ls ${JOBS_DIRECTORY}/* > all_jobs.txt

#count the number of lines on each file
impcount=`wc -l <  all_jobs.txt`
echo ${impcount} > xxx.tmp
NIMPS=`perl -n -e '@splitline=split(/ /,$_); $splitline[1]=~s/ //g; print $splitline[0] ."\n"; ' xxx.tmp`;
rm -f xxx.tmp

#rename jobs postings into something more readable
RUN=1
until [ $RUN -gt ${NIMPS} ]
do
	FNAME=`sed -n $RUN'p' all_jobs.txt`	
	mv ${FNAME} ${JOBS_DIRECTORY}/${RUN}.txt
	RUN=$(( $RUN + 1 ))
done

#extract the job title and necessary skills
rm -f parsed_job_data.txt
RUN=1
until [ $RUN -gt ${NIMPS} ]
do
	FNAME=`sed -n $RUN'p' all_jobs.txt`	
	perl extract_data.pl ${JOBS_DIRECTORY}/${RUN}.txt >> parsed_job_data.txt
	RUN=$(( $RUN + 1 ))
done

rm -f all_jobs.txt
echo "...parsed job data is @ parsed_job_data.txt"

The skill extraction is actually done on the following PERL script (extract_data.pl).

use strict;

#use the generated HTML as an input argument
my $input_file=$ARGV[0];

my $next_line_is_job_title=0;
my $jobTitle="";

my $next_line_is_area_code=0;
my $areaCode="";

my $next_line_is_skills=0;
my $skills="";

my $next_line_is_company_name=0;
my $companyName="";

#parse through every single line in the file
open (MYFILE, "<$input_file") or die $!;
 while (<MYFILE>) {
 	#chomp;
	
	if ($next_line_is_skills==1)
	{
		$skills=$_;
		$skills =~ s/<dd>|<dt>|<\/dt>|<\/dd>|&nbsp;|\t|\n//g;
		$skills =~ s/\/assets\/images\/detail\/default\/highlite.gif//g;
		$skills =~ s/<b style="background:url\(//g;
		$skills =~ s/\); font-weight: bold;">|<\/b>//g;
		#print "skills=" . $skills . "\n";
		$next_line_is_skills=0;
	}
	
	if ($next_line_is_company_name==1)
	{
		$companyName=$_;
		$companyName =~ s/<dd>|<dt>|<\/dt>|<\/dd>|&nbsp;|\t|\n//g;
		my @splitString=split(/>|</,$companyName);
		$companyName = $splitString[2];
		$next_line_is_company_name=0;

		#print "companyName = " . $companyName . "\n";
	}

	if ($next_line_is_area_code==1)
	{
		$areaCode=$_;
		$areaCode =~ s/<dd>|<\/dd>|<dt>|<\/dt>|&nbsp;|\t|\n//g;
		$next_line_is_area_code=0;

		#print "areaCode=" . $areaCode . "\n";
	}

	if (/<h1 id="jobTitle">/)
	{
		my @split_tmp=split(/>|</,$_);
		$jobTitle=@split_tmp[2];

		#print "jobTitle=" . $jobTitle . "\n";
	}
	
	if (/<dt>Skills:<\/dt>|<dd>Skills<\/dd>/) { $next_line_is_skills=1; }
	
	if (/<dt>Company:|<dd>Company/) { $next_line_is_company_name=1; }
	
	if (/<dt>Area Code:|<dd>Area Code/) { $next_line_is_area_code=1; }
	
 }
 close (MYFILE); 
 
 print "$input_file\t$companyName\t$jobTitle\t$areaCode\t$skills\n";

I then feed the extracted data into a mathematica script; a (readable) pdf version of the Mathematica script is here, and the source is here. In this script, I combined all found skills, ignored skills that were required in less than 30 distinct advertisement (e.g. COBOL and Pascal). Below is the resulting piechart.

Most requested skills in embedded computing jobs.

As expected the most sought after skills in “embedded computing” jobs are C,C++ and Linux. Java, mysql and kernel development is also very strong in demand these days. Surprisingly I saw lots of mobile computing and networking skill requests. However the most surprisingly requested skills is databases (mysql)!

Finally, I am aware that I could have done everything on this post on a single PERL script. However, writing a post about a single script would get tedious very quickly. I also wanted to save the outcome of every single step in my hard-disk so I could perform some additional data tests, without having to connect to dice.com each time.

So here I have a very simple circuit… 5 LEDs connected to 5 different pins (P8_3,P8_4,P8_5,P8_11,P8_12). I would like these LEDs to display a perform a simple pattern whenever the BeagleBone is turned ON. Something like the following video.

The code for this circuit is pretty straight forward and looks like this:

#include "BeagleBone_gpio.h"
#include <signal.h>

#define TRUE 1
#define FALSE 0

int forever = TRUE;

void sighandler(int sig)
{
	printf("CONTROL+C was caught... terminating in a friendly manner\n");
	forever = FALSE;
}

int main()
{
	int i=0;
	int pinID[]={P8_3,P8_4,P8_5,P8_11,P8_12};

	int nbr_selectedPins=sizeof(pinID)/sizeof(*pinID);
	struct gpioID selectedPins[nbr_selectedPins]; 
	pinMode_multiple(selectedPins,pinID,nbr_selectedPins,"out");
	
	unsigned int data_to_write;

    signal(SIGABRT, &sighandler);
	signal(SIGTERM, &sighandler);
	signal(SIGINT,  &sighandler);

	data_to_write=1;	

	while(forever==TRUE)
	{
		for (i=0;i<4;i++)
		{
			digitalWrite_multiple(selectedPins,nbr_selectedPins,data_to_write);
			data_to_write=data_to_write<<1;
			delayms(500);
		}

		for (i=4;i>0;i--)
		{
			digitalWrite_multiple(selectedPins,nbr_selectedPins,data_to_write);
			data_to_write=data_to_write>>1;
			delayms(500);
		}
	}

	cleanup_multiple(selectedPins,nbr_selectedPins);
	
	return 1;
}

To compile and run this code, download my BeagleBone_IO library, save the above code as main.c and type the following commands:

What is neat about my code, is that it is able to trap a signal whenever the user types CONTROL+C and ends the program normally… after performing some generic BeagleBone GPIO cleanup.

Anyway, I would like this program to run whenever my BeagleBone starts up. BeagleBone’s default OS is Angstrom Linux which uses systemd as the initialization scheduler. I am still learning about this, but according to the wikipedia entry and also to Kezhong’s Weblog, this daemon is meant to efficiently express services dependencies, by allowing more services to be launched in parallel at system startup while reducing the need to mess around with many scripts. I used to do this on ubuntu… and it was a mess.

Anyway, I would like to declare my LED blinking program as a service. So I need to execute the following steps:

Step #1: Create a script that will call the program that you wish you launch at boot time
This means we need to create a file in the /usr/bin/ directory with any name. For example, assuming that our compiled LED program is in /home/root/main we can create an executable simple script called myScript.sh with the following lines of code.

Step #2: Create the service file
Create the file /lib/systemd/myScript.service . Actually your service name can be anything you want… not restrained to myScript.service.

Step #3: Write the appropriate service settings
Your /lib/systemd/myScript.service can be for example:

Step #4: Make a symbolic link

Step #5: Make systemd take notice of it, activate the service immediately and enable the service to be started on boot-up

After executing all commands on step 5, the LED program should immediately start. Try switching the BeagleBone power OFF & ON… and you will see the LED program.

How to terminate the script that is constantly running in the background?
If everything went okay, your script in constantly running in the background. To terminate it, you need to kill it. Use the top command and find out the process id (pid) of your looping program. As shown on the screenshot, my main pid is 13.

Listing all jobs using the command "top".

How disable the script from starting up at the boot time?
Just type:

Thanks to Piranha at the official BeagleBoard forum for guidance.

#!/bin/bash
#################################################################################
#Functions (do not modify anything here)
#################################################################################
SEARCH_SKILLS=""

function search_jobs
{
	SEARCH_NAME=${1}
	perl craigslist.pl ${SEARCH_SKILLS}> raw_data.txt

	#create the header of an html file
	echo "<html><title>Job Search Results</title><body>" > job_data.html

	#sort the entire file contents and make sure the best matches are on top
	sort -t! -n -r -k3 raw_data.txt >> job_data.html

	#clean up the file
	perl -p -i -e "s/!/\&nbsp;\&nbsp;\&nbsp;\&nbsp;\&nbsp;\&nbsp;/g" job_data.html

	#terminate the html file
	echo "</body></html>" >> job_data.html
	
	mv job_data.html ${SEARCH_NAME}.html
	rm -f raw_data.txt	
}

#################################################################################
#You may modify your skills below
#################################################################################
SEARCH_SKILLS="embedded, circuit, transistor, VLSI, firmware, RTOS, kernel, MacOSX, JTAG, oscilloscope, HDL, FPGA, Arduino, MSP430, OMAP3540, micro-controllers, microcontrollers, SVN, programmer, Perl, linux, Mathematica, LabVIEW, schematics, Verilog, VHDL"
SEARCH_NAME="engineering"
search_jobs ${SEARCH_NAME}

SEARCH_SKILLS="quantitative, mathematica, finance, programmer, developer, high-frequency, fpga, microcontroller"
SEARCH_NAME="finance"
search_jobs ${SEARCH_NAME}

This script runs with the following command line:

Running the scripts on a macosx terminal window.

After it is done executing it will create two files engineering.html and finance.html, where the candidate can see his best job matches.

Two html files are created with the best job matches

Generated HTML file with the best job matches

#This script fetches the last 2 days new job postings from craigslist that match  
#a specific criteria and reports the URLs that correspond to that match.
#The search criteria comes from the input arguments. The cragislist sites
#are hardwired to the New England area. You may change them by manually 
#altering the variables in Section #3.
#
#Version 0.2 30/march/2012
#Author: Nuno Alves
#
#############################################################################
#Section #1 - load libraries
#############################################################################
use strict;
use POSIX;
use LWP::Simple;

#############################################################################
#Section #2 - input arguments are your skillsets
#############################################################################
my $num_args = $#ARGV + 1;
if ($num_args == 1) { print "You must add some skills as arguments\n"; exit; }

#############################################################################
#Section #3 - defining variables
#############################################################################
#what cragislist sites 
my @search_site=("http://boston.craigslist.org","http://nh.craigslist.org","http://maine.craigslist.org","http://burlington.craigslist.org","http://westernmass.craigslist.org","http://worcester.craigslist.org");

#type what positions you are looking for (egr = engineering, sof = software)
my @positions=("egr","sof","bus","acc");

#this array contains the arguments which are your resume skills
my @skills=@ARGV ;

#############################################################################
#Section #4 - debug code
#############################################################################
#instead of work on every single URL, setting $debug=1, will just scan
#two webpages
my $debug=0; 
my @debug_urls=("http://boston.craigslist.org/gbs/egr/2902012136.html","http://boston.craigslist.org/bmw/egr/2929181526.html","http://boston.craigslist.org/gbs/egr/2926742528.html");

#############################################################################
#Section #5 - subroutines for collecting craigslist data
#############################################################################
sub collect_job_posting_http
{
	my $url=$_[0];
	my $content = get $url;
	
	#print $content . "\n";
	my @splitcontents=split(/<h4 class=\"ban\"/,$content);
	my $size_splitcontents=@splitcontents;
	
	my @url_data=();
	for (my $i=1 ; $i<$size_splitcontents ; $i++)
	{
		#just want the last 2 days of postings
		if ($i<3) 
		{
			#print "============\n\n\n";
			#print $splitcontents[$i] . "\n";
		
			#get all the posting urls for this particular day
			my @postingdata=split(/<p><a href=\"|\">/,$splitcontents[$i]);
	
			for (my $j=0; $j<@postingdata ; $j++)
			{	
				#print ">>[$j]>>" . $postingdata[$j] . "<<<\n";
				if ($postingdata[$j]=~m/^http/) 
				{
					push(@url_data,$postingdata[$j]);
				}
			}
		}
	}

return(@url_data);	
}


sub extract_date
{	
	my @url_data=$_[0];
	my @date_data=split(/Date: 2012-|EDT<br>/,$url_data[0]);	

	return("2012-" . $date_data[1]);
}

#############################################################################
#Section #6 - main program: collecting http data for each job posting
#############################################################################
my @urls=();


if ($debug == 0)
{
	for (my $k=0;$k<@search_site;$k++)
	{
		for (my $z=0;$z<@positions;$z++)
		{
			my $base_url=$search_site[$k]."/".$positions[$z];
			my @tmp_data=collect_job_posting_http $base_url;
			push(@urls,@tmp_data);
		}
	}
}
else
{
	@urls=@debug_urls;
}

#foreach (@urls)
#{
#	print $_ . "\n";
#}


#############################################################################
#Section #7 - check if each posting matches at least one skill
#############################################################################
my @matched_skills=();
my @skill_type=();
my @post_date=();

for (my $i=0 ; $i<@urls ; $i++)
{
	my $url=$urls[$i];
	my $content = get $url;
	my $counter=0;
	my $date;
	
#	print $url . "\n";
#	print $content . "\n";
	
	my $skill_type_desc="";

	for (my $k=0; $k<@skills ; $k++)
	{
		if ($content =~ m/$skills[$k]/i) 
		{ 
			$counter++; 
			$skill_type_desc = $skill_type_desc . $skills[$k] . " ";
		}	
	}
	push(@matched_skills,$counter);
	push(@skill_type,$skill_type_desc);
	push(@post_date,extract_date($content));
}

#############################################################################
#Section #8 - print results to the screen
#############################################################################
for (my $i=0; $i < @matched_skills ; $i++)
{
	if ($matched_skills[$i]>0)
	{ 
		print "<li><a href=\"$urls[$i]\">site #$i\<\/a\>" . "!" . $post_date[$i] . "!" . $matched_skills[$i] . "!" . $skill_type[$i] . "\n"; 
	}
}