<!-- mobian-agent-page publisher="dailydev" canonical="https://daily.dev/posts/sysadmin-devops-and-web-scraping---solveit-lesson-6-vwcspfmdj" -->

---
title: Sysadmin, devops, and web scraping - Solveit lesson 6
description: This livestreamed coding lesson shows how to package Python code into a pip-installable library using nbdev, then shifts to web scraping basics. Jeremy Howard...
canonical: https://daily.dev/posts/sysadmin-devops-and-web-scraping---solveit-lesson-6-vwcspfmdj
twitter:card: summary_large_image
twitter:site: @dailydotdev
og:type: website
og:site_name: daily.dev
og:title: Sysadmin, devops, and web scraping - Solveit lesson 6 | daily.dev
og:description: This livestreamed coding lesson shows how to package Python code into a pip-installable library using nbdev, then shifts to web scraping basics. Jeremy Howard...
og:url: https://daily.dev/posts/sysadmin-devops-and-web-scraping---solveit-lesson-6-vwcspfmdj
og:image: https://api.daily.dev/og/posts/vWCSPfmDJ.png
og:image:alt: Sysadmin, devops, and web scraping - Solveit lesson 6
og:image:width: 1200
og:image:height: 630
og:locale: en
---

> ## Documentation Index
> Fetch the complete documentation index at: https://daily.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# Sysadmin, devops, and web scraping - Solveit lesson 6

**[Jeremy Howard](https://daily.dev/sources/howardjeremyp)** · 136 min read · 0 upvotes · 0 comments

## Summary

This livestreamed coding lesson shows how to package Python code into a pip-installable library using nbdev, then shifts to web scraping basics. Jeremy Howard builds a library called fhj daisy live, using the 'gh repo create' command, 'nbdev new', and 'nbdev export' to turn notebooks into a pypi package with autogenerated documentation and GitHub Actions CI. He also publishes it to PyPI with 'nbdev_pypi'. The second half covers scraping a bus timetable page with httpx to fetch HTML and BeautifulSoup to parse it, finding a table with id 'time-table' to extract specific data.

## Full article

daily.dev links to this article rather than hosting it. Read it at the original source: <https://www.youtube.com/watch?v=Ap4DmaB0gLY>

## Questions this post answers

### How do I turn my Jupyter or nbdev notebooks into a pip-installable Python package?

Use nbdev: create a GitHub repo with 'gh repo create', run 'nbdev_new' inside it to scaffold the project, write code in notebooks marking exportable cells with '#|export', then run 'nbdev_export' to generate the .py files. Running 'nbdev_pypi' uploads the package to PyPI once you've registered and added your token to .pypirc.

_Developers packaging their first library can revisit this nbdev workflow on daily.dev before shipping to PyPI._

### What's the difference between httpx and the requests library for fetching web pages in Python?

httpx behaves much like the well-known requests library but is more modern and supports additional features, such as async requests and HTTP/2. There is not really any reason to use requests for new projects anymore, since httpx can do a simple GET request and return page contents just as easily.

_Anyone choosing an HTTP client for a scraping project can compare these trade-offs on daily.dev._

### How do I extract a specific table from an HTML page using BeautifulSoup in Python?

Fetch the page HTML with httpx, parse it with BeautifulSoup to get a 'soup' object, then locate the element by its ID, for example 'soup.find(id="time-table")' when the target table has id 'time-table'. From there you can drill into the table's headers and rows to pull out the specific column or data you need, step by step.

_Developers scraping structured data from messy government or transit sites can revisit this BeautifulSoup technique on daily.dev._

## Similar posts on daily.dev

- [Python Web Scraping Quiz – Real Python](https://daily.dev/posts/python-web-scraping-quiz-real-python-uqmj7bagx) · Real Python · 0 upvotes · 0 comments

---

Tags: [#python](https://daily.dev/tags/python), [#ai-coding](https://daily.dev/tags/ai-coding), [#crawling](https://daily.dev/tags/crawling)

[View this post on daily.dev](https://daily.dev/posts/sysadmin-devops-and-web-scraping---solveit-lesson-6-vwcspfmdj)

```json
{"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]}
{"@context":"https://schema.org","@type":"TechArticle","headline":"Sysadmin, devops, and web scraping - Solveit lesson 6","url":"https://daily.dev/posts/sysadmin-devops-and-web-scraping---solveit-lesson-6-vwcspfmdj","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/sysadmin-devops-and-web-scraping---solveit-lesson-6-vwcspfmdj"},"datePublished":"2026-11-17T00:30:25.335Z","dateModified":"2026-11-17T00:33:36.227Z","description":"This livestreamed coding lesson shows how to package Python code into a pip-installable library using nbdev, then shifts to web scraping basics. Jeremy Howard...","image":"https://i.ytimg.com/vi/Ap4DmaB0gLY/sddefault.jpg","thumbnailUrl":"https://i.ytimg.com/vi/Ap4DmaB0gLY/sddefault.jpg","isAccessibleForFree":true,"articleSection":"Jeremy Howard","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Jeremy Howard","logo":"https://media.daily.dev/image/upload/s--rqXYQTz2--/f_auto/v1724338874/logos/howardjeremyp","url":"https://daily.dev/sources/howardjeremyp"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/sysadmin-devops-and-web-scraping---solveit-lesson-6-vwcspfmdj","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"python,ai-coding,crawling","timeRequired":"PT136M","video":{"@type":"VideoObject","name":"Sysadmin, devops, and web scraping - Solveit lesson 6","description":"This livestreamed coding lesson shows how to package Python code into a pip-installable library using nbdev, then shifts to web scraping basics. Jeremy Howard...","thumbnailUrl":"https://i.ytimg.com/vi/Ap4DmaB0gLY/sddefault.jpg","uploadDate":"2026-11-17T00:30:25.335Z","duration":"PT136M","url":"https://api.daily.dev/r/vWCSPfmDJ","embedUrl":"https://www.youtube.com/embed/Ap4DmaB0gLY"}}
{"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Jeremy Howard","item":"https://daily.dev/sources/howardjeremyp"},{"@type":"ListItem","position":3,"name":"Sysadmin, devops, and web scraping - Solveit lesson 6"}]}
{"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/sysadmin-devops-and-web-scraping---solveit-lesson-6-vwcspfmdj#faq","mainEntity":[{"@type":"Question","name":"How do I turn my Jupyter or nbdev notebooks into a pip-installable Python package?","acceptedAnswer":{"@type":"Answer","text":"Use nbdev: create a GitHub repo with 'gh repo create', run 'nbdev_new' inside it to scaffold the project, write code in notebooks marking exportable cells with '#|export', then run 'nbdev_export' to generate the .py files. Running 'nbdev_pypi' uploads the package to PyPI once you've registered and added your token to .pypirc. Developers packaging their first library can revisit this nbdev workflow on daily.dev before shipping to PyPI."}},{"@type":"Question","name":"What's the difference between httpx and the requests library for fetching web pages in Python?","acceptedAnswer":{"@type":"Answer","text":"httpx behaves much like the well-known requests library but is more modern and supports additional features, such as async requests and HTTP/2. There is not really any reason to use requests for new projects anymore, since httpx can do a simple GET request and return page contents just as easily. Anyone choosing an HTTP client for a scraping project can compare these trade-offs on daily.dev."}},{"@type":"Question","name":"How do I extract a specific table from an HTML page using BeautifulSoup in Python?","acceptedAnswer":{"@type":"Answer","text":"Fetch the page HTML with httpx, parse it with BeautifulSoup to get a 'soup' object, then locate the element by its ID, for example 'soup.find(id=\"time-table\")' when the target table has id 'time-table'. From there you can drill into the table's headers and rows to pull out the specific column or data you need, step by step. Developers scraping structured data from messy government or transit sites can revisit this BeautifulSoup technique on daily.dev."}}]}
```

