--- title: Wikipedia page views as a movie popularity proxy | daily.dev description: A live coding session building a movie tracker app that uses Wikipedia as a backend instead of a broken movie API. The developer implements Wikipedia search,... canonical: https://daily.dev/posts/wikipedia-page-views-as-a-movie-popularity-proxy-dsry7rbnf twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Wikipedia page views as a movie popularity proxy | daily.dev og:description: A live coding session building a movie tracker app that uses Wikipedia as a backend instead of a broken movie API. The developer implements Wikipedia search,... og:url: https://daily.dev/posts/wikipedia-page-views-as-a-movie-popularity-proxy-dsry7rbnf og:image: https://api.daily.dev/og/posts/dSrY7rBnf.png og:image:alt: Wikipedia page views as a movie popularity proxy og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Wikipedia page views as a movie popularity proxy **[sphaerophoria](https://daily.dev/sources/sphaerophoria)** · 113 min read · 1 upvotes · 0 comments ## Summary A live coding session building a movie tracker app that uses Wikipedia as a backend instead of a broken movie API. The developer implements Wikipedia search, parses movie infobox data (title, year, poster), handles HTTPS requests with TLS, manages async state machines, and adds Wikipedia page view stats as a movie popularity proxy. Key challenges include URL percent-encoding, Wikipedia API rate limiting, parsing edge cases in wikitext, and debugging TCP/DNS/SSL connection issues. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### How can I get movie popularity data from Wikipedia instead of a paid movie database API? Movie popularity can be approximated using the Wikimedia pageviews API, which returns monthly view counts per article going back to around 2015 for most pages. Summing monthly views for a movie's Wikipedia page gives a rough popularity proxy, though older or heavily bot-scraped articles can produce skewed numbers. _Developers comparing data sources for ranking or recommendation features can track write-ups like this on daily.dev._ ### Why does the Wikimedia API rate limit unauthenticated requests and how do I avoid getting throttled? Wikimedia limits requests with no identifying characteristics beyond IP address to 10 per minute, specifically to reduce load from high-volume commercial customers. Sending a compliant User-Agent header containing contact information, such as an email or a project name, raises the effective limit to around 200 requests per minute. _Anyone building scrapers or integrations against public APIs can find practical rate-limit workarounds like this on daily.dev._ ### How do you extract structured film data like release year and poster from a Wikipedia page? Structured film data can be pulled from the wikitext infobox film template embedded in a Wikipedia article's raw content, accessible via the Wikipedia REST API. Fields such as release date and poster filename appear as key-value pairs inside double curly braces, though parsing requires handling quirks like lowercase date keys, embedded HTML line breaks in titles, and non-numeric template parameters. _Developers parsing wikitext or building custom data pipelines can follow practical breakdowns like this on daily.dev._ ## Similar posts on daily.dev - [Xikipedia](https://daily.dev/posts/xikipedia-tngv2svkb) · Hacker News · 1 upvotes · 0 comments - [Wikipedia is getting in on the yearly wrapped game](https://daily.dev/posts/wikipedia-is-getting-in-on-the-yearly-wrapped-game-qyzxerykj) · The Verge · 0 upvotes · 0 comments - [WikiFlix shows us what Netflix would have been like 100 years ago](https://daily.dev/posts/wikiflix-shows-us-what-netflix-would-have-been-like-100-years-ago-zytku0xdy) · TechCrunch · 0 upvotes · 0 comments - [I Thought This Wikipedia App for Linux was Pointless \(I Was Wrong\)](https://daily.dev/posts/i-thought-this-wikipedia-app-for-linux-was-pointless-i-was-wrong--bnd3jqjkj) · It's Foss · 1 upvotes · 0 comments --- Tags: [#crawling](https://daily.dev/tags/crawling), [#zig](https://daily.dev/tags/zig) [View this post on daily.dev](https://daily.dev/posts/wikipedia-page-views-as-a-movie-popularity-proxy-dsry7rbnf) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Wikipedia page views as a movie popularity proxy","url":"https://daily.dev/posts/wikipedia-page-views-as-a-movie-popularity-proxy-dsry7rbnf","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/wikipedia-page-views-as-a-movie-popularity-proxy-dsry7rbnf"},"datePublished":"2026-07-18T22:37:44.965Z","dateModified":"2026-09-13T19:28:20.625Z","description":"A live coding session building a movie tracker app that uses Wikipedia as a backend instead of a broken movie API. The developer implements Wikipedia search,...","image":"https://i.ytimg.com/vi/JU-tWRjhS-Y/sddefault.jpg","thumbnailUrl":"https://i.ytimg.com/vi/JU-tWRjhS-Y/sddefault.jpg","isAccessibleForFree":true,"articleSection":"sphaerophoria","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"sphaerophoria","logo":"https://media.daily.dev/image/upload/s--dgPUg2T7--/f_auto/v1728806109/logos/sphaerophoria","url":"https://daily.dev/sources/sphaerophoria"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/wikipedia-page-views-as-a-movie-popularity-proxy-dsry7rbnf","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":1},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"crawling,zig","timeRequired":"PT113M","video":{"@type":"VideoObject","name":"Wikipedia page views as a movie popularity proxy","description":"A live coding session building a movie tracker app that uses Wikipedia as a backend instead of a broken movie API. The developer implements Wikipedia search,...","thumbnailUrl":"https://i.ytimg.com/vi/JU-tWRjhS-Y/sddefault.jpg","uploadDate":"2026-07-18T22:37:44.965Z","duration":"PT113M","url":"https://api.daily.dev/r/dSrY7rBnf","embedUrl":"https://www.youtube.com/embed/JU-tWRjhS-Y"}} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"sphaerophoria","item":"https://daily.dev/sources/sphaerophoria"},{"@type":"ListItem","position":3,"name":"Wikipedia page views as a movie popularity proxy"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/wikipedia-page-views-as-a-movie-popularity-proxy-dsry7rbnf#faq","mainEntity":[{"@type":"Question","name":"How can I get movie popularity data from Wikipedia instead of a paid movie database API?","acceptedAnswer":{"@type":"Answer","text":"Movie popularity can be approximated using the Wikimedia pageviews API, which returns monthly view counts per article going back to around 2015 for most pages. Summing monthly views for a movie's Wikipedia page gives a rough popularity proxy, though older or heavily bot-scraped articles can produce skewed numbers. Developers comparing data sources for ranking or recommendation features can track write-ups like this on daily.dev."}},{"@type":"Question","name":"Why does the Wikimedia API rate limit unauthenticated requests and how do I avoid getting throttled?","acceptedAnswer":{"@type":"Answer","text":"Wikimedia limits requests with no identifying characteristics beyond IP address to 10 per minute, specifically to reduce load from high-volume commercial customers. Sending a compliant User-Agent header containing contact information, such as an email or a project name, raises the effective limit to around 200 requests per minute. Anyone building scrapers or integrations against public APIs can find practical rate-limit workarounds like this on daily.dev."}},{"@type":"Question","name":"How do you extract structured film data like release year and poster from a Wikipedia page?","acceptedAnswer":{"@type":"Answer","text":"Structured film data can be pulled from the wikitext infobox film template embedded in a Wikipedia article's raw content, accessible via the Wikipedia REST API. Fields such as release date and poster filename appear as key-value pairs inside double curly braces, though parsing requires handling quirks like lowercase date keys, embedded HTML line breaks in titles, and non-numeric template parameters. Developers parsing wikitext or building custom data pipelines can follow practical breakdowns like this on daily.dev."}}]} ```