Mining + scraping · Durham, NC
Data mining and web scraping for Durham businesses
Durham owners sell through markets, restaurants, holiday shopping days and online orders. We pull those records together with a source on every row. When you need an outside check, we collect public prices and review counts for comparable Durham businesses.
Mining Durham market, storefront and online orders
The Durham Farmers' Market sells only items made by the people selling them, within a set distance, and runs Saturdays all year. A producer there often adds a web shop and wholesale orders from restaurants downtown.
Those three revenue streams appear in different apps. We combine them with a column for channel, and you see which one grows.
Wholesale buyers often reorder on a rhythm, such as every second Tuesday. Once those orders sit in a dated table with the market and web sales, the rhythm becomes visible and you can plan batches around it.
Restaurants and shops near American Tobacco and Main Street
Downtown has nearly one hundred fifty restaurants and bars, so competing for attention is the daily job. Reservation tools, delivery apps and the register each hold a slice of the same customers.
We line up the exports by date and tag every row with its source, which lets an owner see which night, channel and menu item drives revenue.
Late-night traffic from the ballpark and theater differs from lunch traffic by office workers. A table that keeps the time of day, where your register records it, lets you price and staff each differently.
Scraping public Durham listings for a fair comparison
A small script reads public pages from comparable businesses, noting price, rating, review count and visit date. For a restaurant, that might be lunch plates. For a studio, class drop-in rates.
We keep to public pages and each site's terms, and we do not touch login areas.
We hand you the script's output as a plain sheet. You can open it in any spreadsheet app and sort it yourself.
Holiday calendar planning with your own data
The downtown holiday calendar lists a tree lighting, a shopping day and a parade. Last year's records, sorted by date, tell you what each did for you before you plan staffing for this one.
The same table feeds cleaning, analysis and charts later on.
Questions from Durham owners
I sell wholesale and at the market. Can both go in one table?
Yes. We add a channel column for wholesale, market and online. Totals and per-channel numbers come from the same rows.
What do I need to bring to a first call?
A list of the apps and files where your records live. Messy is fine. We start from whatever exists.
Is scraping allowed for a Durham restaurant?
Collecting facts from public pages, such as listed prices and review counts, within each site's terms, is the only kind we do.
Sources for the facts on this page: Durham Farmers' Market · Downtown Durham, Inc.
Tell us what your Durham records look like
Book a free call of 15 or 30 minutes. We will say where we would begin and what the first table would show.