Skip to content
AskRobots
Browse News How results work Settings

Credits

AskRobots is built on open data and open-source software made by many other people. This page credits them and gives their licences. Listing a source here doesn't mean it endorses AskRobots.

Data

Lists and datasets that help us decide what to crawl, how to rank it and how to read it.

Tranco
A research list of popular websites: which well-known sites to crawl first and how much.
Credit: Victor Le Pochat, Tom Van Goethem, Samaneh Tajalizadehkhoob, Maciej Korczyński and Wouter Joosen. "Tranco: A Research-Oriented Top Sites Ranking Hardened Against Manipulation." NDSS 2019.
Tranco asks users of the list to cite this paper.
Majestic Million
Sites ranked by the links pointing to them: one of our site-importance signals.
Licence: CC BY 3.0 · Credit: Majestic Million by Majestic
Chrome UX Report (CrUX)
Which sites Chrome users visit most, in rank groups: a site-importance signal.
Licence: CC BY 4.0 · Credit: CrUX datasets by Google
Common Crawl web graph
Link-based site rankings (harmonic centrality and PageRank): a site-importance signal.
Licence: Common Crawl terms of use · Credit: Common Crawl
Wikipedia external links
How often English Wikipedia articles link to a site (counts only, no article text).
Licence: Wikimedia dumps licensing · Credit: Wikipedia, from the Wikimedia dumps
Wikidata
The official websites of well-known organisations, and their names.
Licence: CC0 1.0 (public domain) · Credit: Wikidata
No credit is required; we give it gladly.
DMOZ, the Open Directory Project
Titles and descriptions of sites we have not visited yet, and the start of our topic directory. The directory closed in 2017; its listings are from 1998 to 2017.
Credit: DMOZ, the Open Directory Project
Public Suffix List
Telling a site apart from the domain it sits under (lib.example.edu belongs to example.edu).
Licence: MPL 2.0 · Credit: Public Suffix List (Mozilla)
fastText language identification (lid.176)
Recognising the language of each page we index.
Licence: CC BY-SA 3.0 · Credit: Armand Joulin, Edouard Grave, Piotr Bojanowski and Tomas Mikolov. "Bag of Tricks for Efficient Text Classification." 2016.
A model published by Facebook AI Research (Meta). We use it as published; we do not change or share it.
CMU Pronouncing Dictionary
How words sound, for "Did you mean" spelling suggestions.
Licence: BSD-style · Credit: Carnegie Mellon University
Peter Norvig's English word counts
How common each English word is, for spelling suggestions (counts from the Google Web Trillion Word Corpus).
Credit: Peter Norvig, norvig.com/ngrams
english-words
A list of English words for spelling suggestions.
Licence: Unlicense (public domain) · Credit: dwyl/english-words
Webster's Second International word list
More English words for spelling suggestions.
Licence: Public domain · Credit: Webster's Second International Dictionary (1934)

Local results

The Houston businesses shown above some results.

OpenStreetMap
Businesses and places in the Houston area: names, kinds, addresses, opening hours and websites (the Texas extract from Geofabrik).
Licence: ODbL 1.0 · Credit: © OpenStreetMap contributors
The place data we derive from OpenStreetMap stays under the ODbL 1.0. Ask us for a copy.
Texas Department of Licensing and Regulation (TDLR)
Which Houston-area companies hold an active state licence (for example A/C contractors and electricians), from the TDLR list on the Texas Open Data Portal.
Credit: Texas Department of Licensing and Regulation
Public information; no licence stated. Check any licence on TDLR's own license search before relying on it.
Texas State Board of Plumbing Examiners (TSBPE)
Which Houston-area plumbing companies hold an active licence, from the board's free licensee list.
Credit: Texas State Board of Plumbing Examiners
Public information; no licence stated. The board lists it as current to the previous work day, without guarantees.

Quick answers

Sources of the short answers shown above the results.

National Weather Service
Current conditions and forecasts for US places.
Licence: Public domain (US government work) · Credit: Forecast from the National Weather Service
US Census Bureau Gazetteer files
Finding US cities, towns and ZIP codes on the map (with Census population estimates to pick the likely place).
Licence: Public domain (US government work) · Credit: US Census Bureau
Wiktionary
Word definitions.
Licence: CC BY-SA 4.0 · Credit: From Wiktionary, the free dictionary
Definitions we show from Wiktionary are shared under CC BY-SA 4.0 as well.

Software

Open-source software AskRobots runs on. Thank you to everyone who makes it.

Python
The language AskRobots is written in.
Licence: PSF License
PostgreSQL
Our database.
Licence: PostgreSQL License
PgBouncer
Database connection pooling.
Licence: ISC License
Tantivy
The full-text search index.
Licence: MIT License
Flask
The web application.
Licence: BSD-3-Clause
Gunicorn
Serves the web application.
Licence: MIT License
nginx
The web server in front of it all.
Licence: BSD-2-Clause
fastText
Runs the language identification model.
Licence: MIT License

Fonts

Newsreader
Headings. By Production Type.
Licence: SIL Open Font License 1.1 · Credit: Copyright 2020 The Newsreader Project Authors

AI

Some features, such as topic cards and related searches, are written by open-weight language models that run on our own hardware. Today no search is sent to an outside AI service.

Something missing?

If we use your work and haven't credited it properly, please email us and we'll fix it. More about us: About AskRobots.

AskRobots An independent web search index Browse topics How results work Advanced search Search help Make it your default
For site owners Help promote AskRobots Add a page API Privacy Terms About Credits Contact