DeepScreen (AI Hiring Platform, Final Year Project)
End-to-end AI hiring platform, from résumé ingestion to offer decision: Rust-native ONNX inference for a three-stage shortlist (HNSW, cross-encoder, BM25), a face-verification gate, an AI avatar interview with offline TTS, and a Tauri desktop proctor that watches the camera and the OS.
Problem
Hiring breaks in three places at once. Keyword-matching applicant tracking systems reject qualified people before a human ever reads their résumé. The AI-augmented ones that try to fix that usually bolt Python microservices onto the side, paying for interpreter overhead, garbage-collection pauses and serialisation on every request. And remote assessments run in a browser tab that can't see anything outside its own sandbox: an AI copilot in another window, a remote-desktop session, or a different person sitting the test.
DeepScreen is our final year project at the Information Technology University, Lahore, built by Bazil Suhail, Abdullah Masood and M. Ahmad Saleem under the supervision of Dr. Mehwish Ghafoor. It takes a candidate from résumé upload to recruiter decision in one system, with every stage instrumented and every decision logged.
Architecture
Résumé pipeline. EdgeParse, a pure-Rust converter, turns the PDF into Markdown. Groq then runs two passes: one sorts the text into canonical sections, the next extracts typed fields (skills, experience, projects, summary, education, links). Links are taken from regex first and merged with what the LLM found. Four sections are embedded separately as 768-dimensional vectors with an INT8 all-mpnet-base-v2 model, each into its own Qdrant collection. PostgreSQL holds the four vector IDs, the extracted text, links and the original PDF's Cloudinary URL, so one query replaces four round trips.
Shortlisting. Ranking runs in three stages against the job description:
- Four parallel HNSW searches in Qdrant over gRPC build a pool of the top 2K, weighting experience 0.35, skills 0.30, projects 0.20 and summary 0.15 (cosine, m=16, ef_construct=200, ef=128, Recall@10 of 0.93).
- An INT8 ms-marco-MiniLM-L12-v2 cross-encoder re-scores the pool and narrows it to the top K. Each candidate's four (job field, résumé field) pairs go through as one batched tensor and a single forward pass, about 3× faster than four sequential calls.
- BM25 (k1=1.2, b=0.75) over the raw text reorders those K, catching exact requirements that semantic similarity smooths away.
Scores are min-max normalised and blended as 0.65 × cross-encoder + 0.20 × vector + 0.15 × BM25.
Face gate. At sign-up, MediaPipe finds the face, the crop is normalised to 112×112, and an InsightFace w600k_mbf (MobileFaceNet with ArcFace loss) model produces a 512-d embedding stored in Qdrant under the user's ID. Before every assessment stage the Tauri client re-embeds the face, fetches the stored embedding from Qdrant by ID, and computes the cosine similarity in Rust, with a match at 0.6 or above. A failed check blocks entry to the stage and notifies the recruiter. Detection and embedding both run inside the desktop app, so raw frames never leave the device and no face image is stored.
Assessment stages. Shortlisted candidates move through a gated sequence, so recruiter time is spent only on people who have already proven something:
- Coding test: Judge0 CE runs the code in a sandbox while the Tauri client proctors.
- AI avatar interview: a Three.js and Ready Player Me avatar with lip sync asks adaptive questions generated by Llama 3 70B on Groq from the gap between the job and the résumé. Speech comes from Supertonic 3 through sherpa-onnx, offline and CPU-only, and answers come back through the Web Speech API.
- Live interview: a LiveKit SFU room for the candidate, the recruiter, a technical interviewer and HR.
The recruiter gets an evidence pack at the end: résumé summary, coding scores, interview transcript, proctoring log and an integrity confidence score.
Tech decisions & trade-offs
Why inference runs inside the Rust process
The usual shape is a web API calling a separate Python model server. Every request then pays for JSON serialisation, a network hop and the Python runtime's memory and GIL. DeepScreen loads the ONNX models directly into the Rust/Rocket API through the ort crate, so the embedder and cross-encoder run in the same process that handles the request. The cost is a thinner ML library ecosystem and longer compile times than Python.
Why INT8 quantization
The embedder drops from 436 MB to 110 MB and the cross-encoder from 134 MB to 34 MB, 570 MB to 144 MB combined (a 74.7% reduction) for a quality loss under 0.8 percentage points. The sizes come from the published model files, and the quality figure from published INT8 results for these models rather than our own measurement. Either way, it's the difference between a model pair that needs a dedicated box and one that fits next to the API on modest hardware.
Why Qdrant over gRPC
A published benchmark on Qdrant 1.18.1 with 5,000 points puts gRPC at 1.11 ms p50 and 1.38 ms p95, against 2.40 ms and 2.98 ms over REST, and Protobuf payloads run about 75% smaller than JSON. With four searches per shortlist request, that saving compounds.
Why a desktop proctor instead of a browser tab
A browser proctor can watch the webcam, but it can't see what else is running on the machine. The Tauri client runs three streams in parallel: MediaPipe Face Mesh with OpenCV solvePnP for head pose and gaze, a YOLOv8s INT8 model for phones, tablets and extra people, and a process monitor polling every two seconds for AI copilots, remote-desktop tools, browser automation and screen overlays. The signals feed a weighted integrity score with per-signal thresholds and consecutive-frame requirements, so a single glance doesn't raise an alert. Tauri keeps the client under 10 MB, compared with 80 to 120 MB for an Electron equivalent.
Integrity flags are evidence, not verdicts
The gated stages do have hard exits: a candidate who fails the coding test or the AI interview doesn't move on. Integrity signals are treated differently. Proctoring flags and face-match failures never reject anyone automatically; they go to the recruiter as evidence to review, because a poor webcam or bad lighting shouldn't cost someone a job. The 0.6 face-match threshold comes from published ArcFace and MobileFaceNet results and hasn't yet been tested for fairness across skin tones and lighting conditions, which we list as a limitation alongside the latency Groq calls add to shortlisting.
The proctoring detection was later rebuilt from scratch in Rust as Vigilo.