Advanced Website Data Scraper / Social Media Software
The challenge
Our mission was to create a tool that saves a lot of time and automates processes that have been done manually before.
12 months timeline
643 commits git stats
Web Software Development
At Coditive we create software, applications, websites and custom tools for businesses. Our solutions allow to stand out on the market, meet individual expectations and save time and money.
For one of our clients that operates in the background screening industry, we’ve created a platform that analyzes and downloads data from different social media public profiles.
Technology
- Puppeteer – Headless Chrome Node.js API
- Node + Express.js
- Socket.IO
- Vue.js + Nuxt.js
Results
We automated all the manual work that the analyst had to do. Instead of spending all day analyzing one person, now a full background screening can be done in a matter of minutes.
How it works
Social media screening of a Subject (a person whom background is analyzed) is based on a three major pillars:
Social media profiles lookup
Based on provided Subject’s data, like a first and last name, analyst using the platform is able to find user’s profiles in social media.
Lookup is based on Google Custom Search engine and our custom algorithm for searching URLs of social media profiles. When profiles are found they are sorted by relevancy and then it’s possible to pick those ones which should be used on two next steps.
Analysis – content lookup in social media profiles
For found profiles can be performed search on Subject’s activities. For example:
- user’s Facebook activity feed could be analyzed,
- Instagram photos descriptions could be checked
- It could be made a lookup on all user’s tweets.
As default we’ve made a list of offensive keywords (but it’s possible to add custom ones) – our algorithm is analysing every user activity and if the offensive keyword is found it makes a screenshot of it.
After that, all Subject’s activities with picked keywords are listed and then it’s possible to generate a PDF report containing all of them.
Social Media Full History Exporter
It’s a feature that allows to generate a GIANT pdf with all Subject’s activity feeds screenshotted. It works on Twitter, Facebook and Instagram.
Need a reliable partner?
We’re here to help. Take the next step.