Search Engine Algorithm Improvement

Improving the skill matching algorithm for MyAmichi's recruiting platform to consider similar and related skills, addressing issues with strict matching and the lack of connection between skills.

Discuss an idea

All Technologies Used

Python
Python
StackOverflow API
StackOverflow API
Linkedin API
Linkedin API

Motivation

To enhance MyAmichi’s search engine algorithm by improving the skill matching process, enabling it to account for similarities between related skills, and recognizing skills that are listed under different names.

Main Challenges

Challenge 1
Inflexible Skill Matching

The algorithm only considered perfectly matched skills, which excluded candidates who had similar skills. For example, a recruiter looking for 'C++' skills would not see candidates with 'C++ embedded' despite the strong similarity. Azati proposed implementing fuzzy matching to evaluate skill similarity and ensure such candidates would be included in the search results.

Challenge 2
Lack of Skill Relationship Awareness

The algorithm didn't account for skill connections across different programming languages and frameworks. For instance, a candidate with 'Django' likely also knows 'Python', but the algorithm did not consider this. Azati’s solution was to incorporate skill relationships by analyzing dependencies between skills and utilizing external data sources like StackOverflow to determine the extent of such connections.

Key Features

  • Fuzzy Matching: Skills are now compared for similarity, even when the skill names vary, ensuring a broader pool of relevant candidates.
  • Skill Dependency Recognition: The algorithm accounts for connections between related skills, such as Django and Python, to improve matching accuracy.
  • Conditional Probability Estimation: Match degree is calculated using conditional probabilities, considering the likelihood of skills existing together based on real-world data.
  • External Data Integration: StackOverflow data was integrated to provide richer skill matching insights and improve the quality of search results.

Our Approach

Enhancing Skill Comparison
Azati introduced fuzzy matching for skills, allowing the algorithm to compare the recruiter's required skill with the candidate's listed skills, ensuring similarities between closely related terms like 'C++' and 'C++ embedded' were accounted for.
Handling Skill Dependencies
The team developed a mechanism to handle asymmetric skill matching, where knowing one skill (e.g., JavaScript) implies knowing another skill (e.g., AngularJS), but not vice versa. This involved estimating the conditional probability of matching skills based on the candidate's profile and the recruiter's criteria.
Mapping LinkedIn Skills to StackOverflow
Azati mapped LinkedIn skills to StackOverflow tags using case-insensitive comparison, abbreviation expansion, lemmatization, and full-text search. This mapping provided a data-driven approach to calculate the match degree between recruiter-entered skills and candidate skills.
Refining Match Accuracy
Using StackOverflow data, Azati calculated the likelihood that a candidate's skill matched the recruiter's required skill, allowing for a more precise and flexible skill-matching algorithm.

Project Impact

The enhanced algorithm has significantly improved the quality of search results, with recruiters finding 27% more relevant candidates than before. This has led to better candidate placements and increased user satisfaction with the MyAmichi platform.

Ready To Get Started

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.