Key takeaways
- Programmatic pages are fine when each one serves a real search need with real inventory or data.
- Only publish and index combinations that have enough listings to be useful, and noindex the rest.
- Control duplicates from filters and sorting with canonicals and consistent URL rules.
- Handle expired listings deliberately, keep sitemaps accurate and watch Search Console’s indexing reports.
On a jobs board, property portal or vehicle marketplace, most organic traffic does not land on individual listings. It lands on pages like “software jobs in Colombo”, “2 bed apartments for rent in Kandy” or “used hybrid cars in Galle”. These category and location combinations match exactly how people search.
Generating these pages automatically from your data is called programmatic SEO. Done well, it is the backbone of every successful marketplace. Done badly, it produces thousands of empty or near-identical pages that search engines ignore, or worse, treat as spam.
What Google actually objects to
In March 2024 Google introduced its “scaled content abuse” spam policy. It targets pages generated at scale primarily to manipulate rankings rather than help users, regardless of whether they were made by people, templates or AI. The key phrase is “little or no value”. Programmatic pages are not the problem. Empty programmatic pages are.
A page listing 47 current software jobs in Colombo, with filters, salary information and links to related searches, is useful. A page for “underwater welding jobs in a small village” with zero listings and a paragraph of generic text is not.
Design the URL structure first
Choose one consistent pattern and stick to it. A typical marketplace hierarchy looks like this:
- /jobs/ for the main hub.
- /jobs/it/ for a category.
- /jobs/it/colombo/ for a category in a location.
- /jobs/colombo/ for all categories in a location.
- /jobs/it/colombo/senior-react-developer-12345/ for an individual listing.
Decide whether category or location comes first and never mix the two orders. Use lowercase, hyphenated, human-readable slugs. Filters that do not represent a real search intent, such as sort order or page size, belong in query parameters rather than the path.
Give listings a durable identity
Titles change after a listing is shared. Include a stable database identifier or non-guessable public ID in the listing URL, and treat the readable title as a slug rather than the identity. When the slug changes, redirect the old version to the current canonical URL. This preserves bookmarks and links without forcing titles to remain frozen.
Model locations once
Store locations as entities with explicit parent-child relationships and aliases. A city should remain the same entity across categories, even when people use several spellings for it. Decide whether a parent-region page includes every child location and apply that rule consistently to inventory counts, breadcrumbs and filters.
Only index pages with enough value
Set a minimum threshold for each type of combination, for example a minimum number of active listings, and apply it automatically:
| Situation | What to do |
|---|---|
| Combination has enough active listings | Publish, index and include in the sitemap |
| Combination has a few listings | Publish for users, add noindex, keep internal links |
| Combination has no listings | Do not generate it, or show related results with noindex |
| Sort order, view mode, tracking parameters | Canonical to the clean URL |
| Deep filter combinations nobody searches for | Noindex, and avoid linking to them in crawlable links |
Make every page genuinely different
Templates are fine. Templates filled with nothing but the city name swapped out are not. Use your data to make each page specific:
- The actual listings, with key details visible in the HTML rather than loaded later by JavaScript.
- Live statistics such as number of listings, typical salary or price range and how recently new items were added.
- Relevant filters and related searches: nearby locations, related categories, popular employers or areas.
- Short, useful context written for that category or location where it adds something, such as typical requirements or local information.
- Accurate titles and meta descriptions built from the data, for example “47 software jobs in Colombo | Updated today”.
Internal linking holds it all together
Search engines discover and understand large sites through links. Build clear paths: hubs link to categories and locations, categories link to their top locations, location pages link to nearby locations and related categories, and every listing links back up through breadcrumbs. Add BreadcrumbList structured data so the hierarchy is explicit.
Handle expired listings deliberately
Marketplaces constantly create and remove pages. Decide on a policy and apply it consistently:
- Recently expired listings can stay live for a while, clearly marked as closed, with links to similar active items. This keeps visitors and any links the page earned.
- Old expired listings with no value can return a 410 (gone) or 404, so search engines drop them.
- Avoid redirecting every expired listing to the homepage; Google may treat mass irrelevant redirects as soft 404s.
- Move expired records out of your main tables or archive them so your active database stays fast.
Sitemaps and crawl budget
Split sitemaps by type (categories, locations, listings) and keep each under the 50,000 URL limit. Only include indexable, canonical URLs, and set lastmod to the date the content actually changed. Google has said crawl budget is mainly a concern for very large sites or sites with content that changes very frequently, which describes many marketplaces. Keeping thin and duplicate URLs out of crawl paths lets search engines spend their effort on pages that matter.
Changing an existing URL structure
Treat a URL migration as a data project rather than a search-and-replace. Export every indexable old URL, map it to one genuinely equivalent destination and test samples from each page type. Update internal links, canonicals and sitemaps to point directly to the new URLs, then keep the redirects for as long as old links and bookmarks may exist.
- Do not redirect every removed listing or category to the homepage; irrelevant mass redirects can be treated as soft 404s.
- Avoid redirect chains by pointing every historical URL directly to the current destination.
- Compare indexed-page counts, crawl errors, traffic and conversions by template after launch.
Measure each page type separately
Group your URLs by type in Search Console and analytics: hubs, categories, locations, category-location combinations and listings. Track impressions, clicks, indexing status and conversions for each group. This shows quickly whether a template change helped or hurt, and which combinations deserve more content or stronger internal links. It also reveals page types that attract crawling but never earn clicks, which are candidates for noindex or removal.
Technical checklist
- One consistent URL pattern for categories, locations and listings.
- Server-rendered or statically generated HTML with listings visible without JavaScript.
- Automatic noindex below your listing threshold; canonical tags for parameters.
- Unique titles, descriptions and headings generated from real data.
- Breadcrumbs with structured data, plus related links on every page.
- A clear policy for expired listings using 410, 404 or a useful closed page.
- Segmented sitemaps with accurate lastmod dates.
- Monthly review of Search Console indexing and performance by page type.
We designed Job Index and the rest of our Index family around exactly these rules, because for a classifieds business, URL structure and indexation are part of the architecture rather than an afterthought. If you are planning or fixing a marketplace, we would be glad to review your structure.
Sources and further reading
- Spam policies: scaled content abuse — Google Search Central
- Build and submit a sitemap — Google Search Central
- Large site owner's guide to managing crawl budget — Google Search Central
Written by the Prestigious Software team
We design, build and run web platforms, SaaS products and mobile apps. About us
Our team reviews each article. Claims based on standards or research link to primary sources; examples are illustrative unless identified as client work.
