Growing Market Demands

Steady market growth and new approaches to managing data and effectively leveraging insights (Machine Learning, Data Lakes, Enterprise Data Hubs), in conjunction with the uncertainty of the government’s approach to H1-Bs, H-4s, and OPT, have created a perfect storm of demand for highly-skilled, US-based data engineers who are well-versed in Big Data and Machine Learning technologies. This rapid growth, speed of change in commercially-proven technology, and high demand for skilled techs has outstripped the available pool of talent. As a result, techs have developed a tendency to creatively embellish their abilities in order to try and open the door into learning the skills they want to have instead of  accurately representing the skills that they have – aka putting the horse before the cart.  

To stay relevant in the rapidly-evolving technology sphere, engineers always want (and need) to learn the technologies that the market demands, and they need a chance to get themselves trained to meet these demands. For many, the preferred method is on-the-job training.  Though employers are often open to on-the-job training, they need to hire the experts who can provide the framework and knowledge base for those who follow. However, as new technologies emerge, building a knowledge-base team presents a catch-22 for the employer as they need the first wave of experts to begin the process, but they do not have the in-house knowledge to vet them effectively.  Consequently, employers reach out to new or third-party talent to help build this base.

Effective Screening

The problem then becomes effectively screening talent. As noted earlier, techs have started embellishing resumes and applications with the buzzwords for skills they want to have instead of the skill they have. This embellishing becomes a challenge for the first line of talent screeners as they rarely have the knowledge base to effectively test the capacity of an individual’s skills for these buzzwords. There is no doubt that many accomplished engineers can bring themselves up to speed in a relatively quick timeframe, and they bank on the idea that their learning curve can be completed before anyone notices that they do not have the expertise they purported to have. Unfortunately, many engineers do not have a realistic ability to self assess how long becoming skilled in Big Data and Machine Learning will take and end up spending valuable time and resources failing to close this gap.

For example, take the recent boom in the demand for “Sparkstars”. Sparkstars are engineers highly skilled in using both Spark and Scala. On a scale of 1 (novice) to 5 (expert), Sparkstars’ Spark/Scala skills easily fall on 5. Most Sparkstars start out as dime-a-dozen Java engineers since most Java engineers can acquire Scala with ease. As such, Java engineers wanting to become Sparkstars will add Spark/Scala qualifications to their resumes even though they haven’t acquired those skillsets yet and hope they can acquire them quickly on the job.

Skills Solutions

So, how can talent recruiters effectively go about testing whether techs actually have these skills or if they only seek to gain these skills?

Our solution to this challenge is in leveraging our proven outside consultants to help foster the proper framework for your data engineering team, vetting full-time hires through our current skilled team members, and provide immediate talent to hit the ground running allowing the power of Big Data and Machine Learning solutions to work for you.