Bemnet Beshah

Projects / Client work

HSP Vendor Finder

I built a Python tool through my consulting agency, Renvane, for a Texas government construction contractor. Given the NIGP trade codes on a state bid, it searches the state’s HUB vendor database, checks each vendor against the trade and against certification dates, and saves the state’s own vendor info sheet as a PDF. It stops at three vendors per trade code.

Problem

Texas state bids need a HUB Subcontracting Plan (HSP): documented, good-faith evidence that HUB-certified subcontractors were sought for each trade on the job. The client’s project director built this by hand. For each NIGP code she searched the Centralized Master Bidders List (CMBL), opened vendors one at a time, read their business descriptions, and checked certifications.

The state audits these plans. A vendor listed under an HVAC code whose own description never mentions HVAC is a liability, so the description check could not be dropped.

The CMBL has no API. Vendor details load by AJAX into a modal, with no URL to visit, and the site rejects headless browsers.

My role

I wrote all of the code (I’m the only author in the git history) and wrote the requirements document and V2 plan from the client’s email and a recorded walkthrough. Renvane delivered it. The client supplied the requirements and the audit rules.

How it works

trade code (class + item, trade name, keywords)
        |
        v
browser-use agent (Claude Sonnet, headed Chrome)
  fills the CMBL search form, stops at the results table
        |
        v   done-callback: one outerHTML snapshot
BeautifulSoup parser -> vendor records -> sort by ZIP tier
        |
        v   for each vendor, in tier order, until 3 saved
open detail modal (JS click) -> read cert dates -> read description
  cert gate -> keyword match -> Claude Haiku fallback
        |
        v   qualified
Chrome printToPDF of the modal -> output/<code>/<company>.pdf
        |
        v
Claude reads each PDF, prints a PASS/FAIL report

Vendors come from the state’s Texas CMBL, searching “VetHUBs On CMBL”. The code does no code matching of its own. The agent types one NIGP class and one item into the state’s form, and the state’s server returns the vendors registered under that pair. Trade keywords come from a hand-written list or, in the interactive runner, from Claude Haiku given the trade name.

Parsing is a plain function over the saved page. Each cell’s text starts with its column name, so I parse by that prefix, not by column position.

Ranking is a three-way sort on ZIP prefix: 787xx first, then 786xx and 785xx, then everything else. Python’s sort is stable, so vendors keep the CMBL’s order inside a tier. There is no score, no tiebreaker and no relevance model. The ranking only decides which vendors get checked first.

Key decisions

An agent for navigation, plain code for data. I used a browser-use agent for the search form, because that part changes cosmetically. The agent’s task text tells it to stop at the results table. Extraction, ranking and qualification are deterministic code, because a model transcribing 15 rows by 13 columns into an audited document will eventually get one wrong.

Keywords first, Haiku second. A substring match settles most vendors for free. Only when it misses do I call Haiku, with a cached system prompt, a 64-token cap and a fixed two-line reply (MATCH: and REASON:). The reason is logged beside each decision. The alternative was Haiku on every vendor, which costs more for cases a substring already settles.

Print the state’s own sheet. I capture the live modal with Chrome’s Page.printToPDF instead of rendering my own template. The attachment proves the vendor exists in the state system, so the state’s layout is the defensible artifact.

Under-deliver visibly. If a trade has fewer than three qualifying vendors, the tool saves what it found and prints N / 3. In the interactive runner, an exception in one trade is caught and the next trade still runs.

What was hard

The session reset. browser-use resets the browser when agent.run() returns, so the results page disappears. All post-search work runs inside register_done_callback, and I pass the tab’s target_id explicitly to CDP because the focus target goes stale after the reset.

Blank PDFs. Isolating the modal with display: none makes Chrome compute a zero-height layout and print a roughly 1 KB empty file. Switching to visibility: hidden fixed it. The validator also rejects files under 2 KB before any API call.

Also: evaluate() returns strings, so JS false arrives as the truthy "False". Every DOM check compares against "True".

Results

  • I have no timing logs for the manual process or for the tool, so I make no claim about hours saved per bid.
  • In one recorded search spike, NIGP 914-50 (HVAC) returned 15 vendors on one page.
  • The repository has 46 test functions across ranking, parsing against a saved CMBL page, and matching with a mocked API client. They run without a browser. One test asserts the keyword fast path makes no API call.

Limitations and next steps

  • It reads the first results page only. If a closer vendor sits on page two, it is missed.
  • Missing or unparseable certification dates pass the expiry gate. A blank description is not short-circuited. It goes to Haiku as an empty string.
  • The PDF validator prints a report and deletes nothing.
  • There is no cross-trade deduplication or flagging, and no summary file beside the PDFs.
  • The HVAC keyword list includes “mechanical”, which is broad.
  • Next: pagination, a hard rule for blank descriptions, and run-level output folders so reruns don’t overwrite.