Robot career guidance: AI may soon be able to analyse your tweets to match you to a job

Imagine yourself graduating from high school, with the world before you.

But now you must decide what career you want to pursue. You hope for a job that will pay the bills, but also one you will enjoy. After all, you will spend a large portion of your waking hours at work.

But how can you make a reliable choice – beyond what your parents might be pushing for, or what your final year results will get you direct entry into.

Our study published today in the Proceedings of the National Academy of Science found different professions attract people with very different psychological characteristics.

When looking for a new career, you might visit a career adviser and answer a set of questions to identify your interests and strengths. These results are used to match you with a set of potential occupations.

However, this method relies on long surveys, and doesn’t account for the fact that many occupations are changing or disappearing as technology transforms the employment landscape.

21st century job search

We wondered if we could develop a data-driven approach to matching a person with a suitable profession, based on psychological traces they reveal online.

Studies have shown people leave traces of themselves through the language they post online and their online behaviours.

Could we analyse this to find out the extent to which people doing the same job shared the same personality traits?

Read more: Employment services aren't working for older jobseekers, jobactive staff or employers

In our research, we identified more than 100,000 Twitter users, each of whom included one of 3,513 job titles in their user profile.

Then, using a tool available through IBM’s cloud-based artificial intelligence engine Watson, and its Personality Insights service, we gave each profile a score across ten personality-related characteristics, based on the language in their posts.

We used a variety of data analytics and machine learning techniques to explore the personality of each of the occupations.

For example, to create the “vocation compass map” we used an unsupervised machine learning algorithm to cluster occupational personality data into twenty distinct clusters, grouping the occupations that were most similar in terms of personality.

An occupational map

Work has long been thought to be more fulfilling if it fits who we are as a person, in terms of our personality, values, and interests.

Our results confirmed this, and we found that different occupations tended to have very different personality profiles.

For instance, software programmers and scientists were generally more open to experiencing a variety of new activities, were intellectually curious, tended to think in symbols and abstractions, and found repetition boring. On the other hand, elite tennis players tended to be more conscientious, organised and agreeable.

Our findings point to the possibility of using data shared on social media to match an individual to a suitable job.

People belonging to different occupations generally have distinct personality traits. This figure shows the digital fingerprints of 1,200 individuals across nine occupations. Each dot corresponds to a user - with people grouped. within their self-identified occupation. Paul X. McCarthy

We used machine learning to cluster more than one thousand roles based on the inferred personality traits of people in those roles.

Read more: Inspire children with good careers advice and they do better at school

We found many similar jobs could be grouped together.

For example, one cluster included different technology jobs such as software programming, web development, and computer science. Another group included gym management, logistic coordination, and concert promotions.

You can explore more with this interactive online map we made.

However, while many of the combinations aligned with existing occupation classifiers (current formal groupings that governments and other organisations use to group jobs together), some clusters included roles not traditionally grouped together.

For instance, cartographers, grain farmers and geologists ended up grouped together and shared similar personality traits to many of the technology professionals.

A data-driven vocation compass

With our results, we explored the idea of building a data-driven vocation compass: a recommendation system that could find the best career fit for someone’s personality.

We built a system that could recommend an occupation aligned to people’s personality traits with over 70% accuracy.

Even when our system was wrong, it wasn’t far off, and pointed to professions with very similar skill sets. For instance, it might suggest a poet becomes a fictional writer.

Professions are quickly changing due to automation and technological breakthroughs. And in our connected, digital world, we leave behind traces of ourselves. Our work has offered one approach to using these traces in a productive way.

Read more: Artificial intelligence may take your job, so political leaders need to start doing theirs

This approach may one day be used to help people find their dream career, or at the very least, better our understanding of the hidden personality dimensions of different roles.