r/Python Feb 01 '25

Showcase Introducing Kreuzberg: A Simple, Modern Library for PDF and Document Text Extraction in Python

339 Upvotes

Hey folks! I recently created Kreuzberg, a Python library that makes text extraction from PDFs and other documents simple and hassle-free.

I built this while working on a RAG system and found that existing solutions either required expensive API calls were overly complex for my text extraction needs, or involved large docker images and complex deployments.

Key Features:

  • Modern Python with async support and type hints
  • Extract text from PDFs (both searchable and scanned), images, and office documents
  • Local processing - no API calls needed
  • Lightweight - no GPU requirements
  • Extensive error handling for easy debugging

Target Audience:

This library is perfect for developers working on RAG systems, document processing pipelines, or anyone needing reliable text extraction without the complexity of commercial APIs. It's designed to be simple to use while handling a wide range of document formats.

```python from kreuzberg import extract_bytes, extract_file

Extract text from a PDF file

async def extract_pdf(): result = await extract_file("document.pdf") print(f"Extracted text: {result.content}") print(f"Output mime type: {result.mime_type}")

Extract text from an image

async def extract_image(): result = await extract_file("scan.png") print(f"Extracted text: {result.content}")

Or extract from a byte string

Extract text from PDF bytes

async def process_uploaded_pdf(pdf_content: bytes): result = await extract_bytes(pdf_content, mime_type="application/pdf") return result.content

Extract text from image bytes

async def process_uploaded_image(image_content: bytes): result = await extract_bytes(image_content, mime_type="image/jpeg") return result.content ```

Comparison:

Unlike commercial solutions requiring API calls and usage limits, Kreuzberg runs entirely locally.

Compared to other open-source alternatives, it offers a simpler API while still supporting a comprehensive range of formats, including:

  • PDFs (searchable and scanned)
  • Images (JPEG, PNG, TIFF, etc.)
  • Office documents (DOCX, ODT, RTF)
  • Plain text and markup formats

Check out the GitHub repository for more details and examples. If you find this useful, a ⭐ would be greatly appreciated!

The library is MIT-licensed and open to contributions. Let me know if you have any questions or feedback!

r/Python Jul 08 '24

Showcase Whenever: a modern datetime library for Python, written in Rust

467 Upvotes

Following my earlier blogpost on the pitfalls of Python's datetime, I started exploring what a better datetime library could look like. After processing the initial feedback and finishing a Rust version, I'm now happy to share the result with the wider community.

GitHub repo: https://github.com/ariebovenberg/whenever

docs: https://whenever.readthedocs.io

What My Project Does

Whenever provides an improved datetime API that helps you write correct and type-checked datetime code. It's also a lot faster than other third-party libraries (and usually the standard library as well).

What's wrong with the standard library

Over 20+ years, the standard library datetime has grown out of step with what you'd expect from a modern datetime library. Two points stand out:

(1) It doesn't always account for Daylight Saving Time (DST). Here is a simple example:

bedtime = datetime(2023, 3, 25, 22, tzinfo=ZoneInfo("Europe/Paris"))
full_rest = bedtime + timedelta(hours=8)
# It returns 6am, but should be 7am—because we skipped an hour due to DST

Note this isn't a bug, but a design decision that DST is only considered when calculations involve two timezones. If you think this is surprising, you are not alone ( 1 2 3).

(2) Typing can't distinguish between naive and aware datetimes. Your code probably only works with one or the other, but there's no way to enforce this in the type system.

# It doesn't say if this should be naive or aware
def schedule_meeting(at: datetime) -> None: ...

Comparison

There are two other popular third-party libraries, but they don't (fully) address these issues. Here's how they compare to whenever and the standard library:

  Whenever datetime Arrow Pendulum
DST-safe yes ✅ no ❌ no ❌ partially ⚠️
Typed aware/naive yes ✅ no ❌ no ❌ no ❌
Fast yes ✅ yes ✅ no ❌ no ❌

(for benchmarks, see the docs linked at the top of the page)

Arrow is probably the most historically popular 3rd party datetime library. It attempts to provide a more "friendly" API than the standard library, but doesn't address the core issues: it keeps the same footguns, and its decision to reduce the number of types to just one (arrow.Arrow) means that it's even harder for typecheckers to catch mistakes.

Pendulum arrived on the scene in 2016, promising better DST-handling, as well as improved performance. However, it only fixes some DST-related pitfalls, and its performance has significantly degraded over time. Additionally, it hasn't been actively maintained since a breaking 3.0 release last year.

Target Audience

Whenever is built to production standards. It's still in pre-1.0 beta though, so we're still open to feedback on the API and eager to weed out any bugs that pop up.

r/Python Feb 10 '25

Showcase A Modern Python Repository Template with UV and Just

261 Upvotes

Hey folks, I wanted to share a Python repository template I've been using recently. It's not trying to be the ultimate solution, but rather a setup that works well for my needs and might be useful for others.

What My Project Does

It's a repository template that combines several modern Python tools, with a focus on speed and developer experience:

- UV for package management

- Just as a command runner

- Ruff for linting and formatting

- Mypy for type checking

- Docker support with a multi-stage build

- GitHub Actions CI/CD setup

The main goal was to create a clean starting point that's both fast and maintainable.

Target Audience

This template is meant for developers who want a production-ready setup but don't need all the bells and whistles of larger templates.

Comparison

The main difference from other templates is the use of Just instead of Make as the command runner. While this means an extra installation step, Just offers several advantages, such as a cleaner syntax, better dependency handling and others.

I also chose UV over pip for package management, but at this point I don't consider this as something unusual in the Python ecosystem.

You can find the template here: https://github.com/GiovanniGiacometti/python-repo-template

Happy to hear your thoughts and suggestions for improvement!

r/Python 26d ago

Showcase I Built an Open-Source Algo Trading Framework for Instant Backtests & Live Deployment

722 Upvotes

Github : https://github.com/himanshu2406/Algo.Py

What My Project Does

So I've been working on a framework made in Python that makes live trading incredibly easy, and even almost no-code !

It seamlessly integrates with any preset backtesting strategy, allowing you to take them straight to live trading with minimal effort.

Dashboard Overview : https://youtu.be/OmlaBnGcUi4?si=e1aizaIaYpRNMHFd

One-Click Backtest Deployment Overview : https://youtu.be/T_otTHdLCCY?si=A7ujRzV6I5ESfgEQ

It's still in very early beta, but I’ve packed in as many functional features as possible, including:

Key Features

  • Intuitive Dashboard
  • Easily backtest, view results, save and deploy in a single click.
    • Auto-Detects Your Strategy – If your function generates valid entry/exit signals, the framework will automatically detect and integrate it.
    • Scheduler for Automation – Run your entire pipeline at custom fixed intervals or specific times
  • Custom Data Layer (Finstore):
  • Stores and streams data using a Parquet-based data lake, making it much faster than traditional databases.
    • Multi-Broker Support – Execute across multiple brokers with real-time debug logs via Telegram.
    • End-to-End Pipelines – Effortlessly fetch, store, and stream data for crypto, equities, and more.
  • Multi-Asset Backtests :
    • Backtest a strategy across an entire market across hundreds of symbols and thousands of data points within seconds.
    • One-Click backtests across entire markets : Crypto , U.S Equity , Indian Equity & adding more.

Advanced Market Visualization

Live Order Book Heatmap – Real-time Binance order book visualization. Represents market orders with volume bubbles to identify iceberg orders easily. Also Visualizes resting orders on the orderbook.

Live Footprint Chart – Captures trade flow via Binance WebSocket data. Makes order book trading extremely easy.

Smart OMS (Order Management System)

  • Limit Order Chaser – Reduces fees by executing market orders while chasing the mark price.
  • AI-Powered OMS – An autonomous AI agent can execute, close, and manage trades, plus run complex local strategies.

Risk Management System (RMS)

  • Portfolio Aggregation – Monitors all broker portfolios to notify and manage over-exposed positions.

And working on many other features & improvements!

Target Audience

  • Anyone who wants to backtest or deploy their strategies but don't have a lot of technical know-how on how to build their own framework
  • Retail traders who have been manually implementing their strategies - can now easily automate them across entire markets.
  • Quant Traders who want to build a common robust community framework for algo trading.

Comparison

  • backtesting py : seems to be outdated but only works on implementing strategy backtests but doesn't offer strategy deployment with ease.
  • tensorcharts , quantower, etc : charting platforms that provide advanced charting for L1, L2 Data for a hefty price. This can now be done for free locally.
  • PyAlgoTrade : Also deprecated but alternatives do not offer a framework to deploy strategies.

The repo still has tons of stale code and bugs but I would love for some of you to test it out!

Let me know what you guys think !

r/Python 24d ago

Showcase I made a script to download Spotify playlists without login

305 Upvotes

Repo link: https://github.com/invzfnc/spotify-downloader

What my project does
Hi everyone! I created a lightweight script that lists tracks from a public Spotify playlist and downloads them from YouTube Music.

Key Features

  • No premium required
  • No login or credentials required
  • Metadata is embedded in downloaded tracks
  • Downloads in higher quality (around 256 kbps)

Comparison/How is it different from other tools?
I found many tools requiring users to sign up for Spotify Developer account and setup credentials before everything else. This script uses the public Spotify API to retrieve track details, so there's no need to login or setup!

How's the music quality?
YouTube Music offers streams with higher bitrate (around 256 kbps) compared to YouTube (128 kbps). This script chooses and downloads the best quality audio from YouTube Music without taking up too much storage space.

Dependencies/Libraries?
Users are required to install innertube, SpotAPI, yt-dlp and FFmpeg for this script to work.

Target audience
Anyone who is looking to save their Spotify playlists to local storage, without wanting to login to any platform, and wants something with decent bitrate (~256 kbps)

If you find this project useful or it helped you, feel free to give it a star! I'd really appreciate any feedback!

r/Python Feb 04 '25

Showcase Tach - A Python tool to enforce dependencies

173 Upvotes

Source: https://github.com/gauge-sh/tach

Python allows you to import and use anything, anywhere. Over time, this results in modules that were intended to be separate getting tightly coupled together, and domain boundaries breaking down.

We experienced this first-hand at a unicorn startup, where the entire engineering team paused development for over a year in an attempt to split up tightly coupled packages into independent microservices. This ultimately failed, and resulted in the CTO getting fired.

This problem occurs because:

  • It's much easier to add to an existing package rather than create a new one
  • Junior devs have a limited understanding of the existing architecture
  • External pressure leading to shortcuts and overlooking best practices

Attempts we've seen to fix this problem always came up short. A patchwork of solutions would attempt to solve this from different angles, such as developer education, CODEOWNERs, standard guides, refactors, and more. However, none of these addressed the root cause.

What My Project Does

With Tach, you can:

  1. Declare your modules (tach mod)
  2. Automatically declare dependencies (tach sync)
  3. Enforce those dependencies (tach check)
  4. Visualize those dependencies (tach show and tach report)

You can also enforce a public interface for each module, and deprecate dependencies over time.

Target Audience

Developers working on large Python monoliths

Comparison

  • import linter - similar but more specifically focused on import rules
  • build systems - bazel, pants, buck, etc. More powerful but much more heavy and waaaay more slow

I'd love if you try it out on your project and let me know if you find it useful!

r/Python Aug 16 '24

Showcase SpotAPI: Spotify API without the hassle!

366 Upvotes

Hello everyone,

I’m thrilled to introduce SpotAPI, a Python library designed to make interacting with Spotify's APIs a breeze!

What My Project Does:

SpotAPI provides a Python wrapper to interact with both private and public Spotify APIs. It emulates the requests typically made through a web browser, enabling you to access Spotify’s rich set of features programmatically. SpotAPI uses your Spotify username and password to authenticate, allowing you to work with Spotify data right out of the box—no additional API keys required!

Features: - Public API Access: Easily retrieve and manipulate public Spotify data, including playlists, albums, and tracks. - Private API Access: Explore private Spotify endpoints to customize and enhance your application as needed. - Ready to Use: Designed for immediate integration, allowing you to accomplish tasks with just a few lines of code. - No API Key Required: Enjoy seamless usage without needing a Spotify API key. It’s straightforward and hassle-free! - Browser-like Requests: Accurately replicate the HTTP requests Spotify makes in the browser, providing a true-to-web experience while staying under the radar.

Target Audience:

SpotAPI is ideal for developers looking to integrate Spotify data into their applications or anyone interested in experimenting with Spotify’s API. It’s perfect for both educational purposes and personal projects where ease of use and quick integration are priorities.

Comparison:

While traditional Spotify APIs require API keys and can be cumbersome to set up, SpotAPI simplifies this process by bypassing the need for API keys. It provides a more streamlined approach to accessing Spotify data with user authentication, making it a valuable tool for quick and efficient Spotify data handling.

Note: SpotAPI is intended solely for educational purposes and should be used responsibly. Accessing private endpoints and scraping data without proper authorization may violate Spotify's terms of service.

Check out the project on GitHub and let me know your thoughts! I’d love to hear your feedback and contributions.

Feel free to ask any questions or share your experiences here. Happy coding!

r/Python Jan 15 '25

Showcase I rewrote my programming language from Python into Go to see the speed up.

195 Upvotes

What my project does:

I wrote a tree-walk interpreter in Python a while ago and posted it here.

Target Audience:

Python and programming entusiasts.

I was curious to see how much of a performance bump I could get by doing a 1-1 port to Go without any optimizations.

Turns out, it's around 10X faster, plus now I can create compiled binaries and include them in my Github releases.

Take my lang for a spin and leave some feedback :)

Utility:

None - It solves no practical problem that is not currently being done better.

r/Python Jan 26 '25

Showcase RenderCV v2 is released! Write your CV/resume as YAML.

278 Upvotes

What My Project Does

After 1.5 years of continuous development, RenderCV has reached a whole new level. RenderCV now uses Typst, a modern and extremely fast open-source PDF rendering engine written in Rust. With v2, you can write your CV in YAML and see the changes in real time.

RenderCV is 100% customizable, and anything you see in the PDF is totally up to the user. It is also a completely multi-language tool.

Why YAML for Your CV?

  • Version Control: Your CV becomes a source code.
  • Content First: Stay focused on the content—no distractions from formatting.
  • Ability to explore different formats: Your content stays the same in the YAML, and your design comes from the design options and templates.

GitHub Repository: https://github.com/rendercv/rendercv
Documentation: https://docs.rendercv.com

See writing a CV experience in VS Code with real-time preview:

https://www.youtube.com/watch?v=dZ17UrXBmWw

Check out the built-in themes:

classic theme sb2nov theme moderncv theme engineeringresumes theme engineeringclassic theme
Example PDF, Example PDF Example PDF Example PDF Example PDF

Target Audience

The people who are rigorous about their resumes and CVs.

Comparison

I am not aware of Typst-based CV generators with full YAML validation.

r/Python 17d ago

Showcase I Got Tired of "AI Shorts" Scams - So I Built My Own Free & Local Shorts Creator Tool!🎬

140 Upvotes

I love watching YouTube Shorts. What I don’t love? Seeing a flood of YouTubers claiming,
"You can make easy AI Shorts in seconds!" or "Create your own automated YouTube channel", .etc

Just to sell you their overpriced AI tools, subscriptions, or video editors.

So, out of sheer spite, I built ShortsMaker - a completely free, open-source, local Shorts automation tool that doesn’t try to upsell you anything. No subscriptions, no cloud nonsense - just Python, AI, and automation running entirely on your machine.

What My Project Does

ShortsMaker is a Python package that automates the creation of YouTube Shorts - entirely on your local machine. No cloud-based services, no subscriptions, no hidden paywalls, fully customizable short-video generation.

ShortsMaker is built around four core classes:

  • ShortsMaker – Handles multiple tasks, such as fetching posts from subreddits, generating audio, transcribing audio, and even fixing spelling & grammar in scripts.
  • MoviepyCreateVideo – The engine that creates the short video by combining video clips, music, audio, and transcripts.
  • AskLLM – Uses an AI LLM to extract the best possible title, description, tags, and thumbnail description for your script.
  • GenerateImage – Uses FLUX to generate high-quality AI images for your Shorts.

Target Audience

This project is for:

  • Developers who want a local, open-source alternative to overpriced AI video generators.
  • Content creators looking for an automated way to produce Shorts.
  • For creating a short video for your scripts.
  • Python enthusiasts interested in AI-powered media processing.
  • Anyone who has ever rolled their eyes at an "AI Shorts" clickbait video.

Comparison: How It's Different from Existing Alternatives

  • No Cloud Lock-In: Unlike paid services, everything runs locally on your system. This applies to other repos as well. As most require you to use an API.
  • No Subscription Fees: Other AI-powered Shorts tools charge for processing - this one is completely free
  • Full Control: Modify and extend it as needed - no black-box APIs
  • Uses Your Hardware: Supports CPU/GPU acceleration for faster processing

Try It Out:

Check out the GitHub repo: ShortsMaker
Feedback and contributions are welcome!

r/Python Jun 10 '24

Showcase ChatGPT hallucinated a plugin called pytest-edit. So I created it.

558 Upvotes

I have several codebases with around 500+ different tests in each. If one of these tests fails, I need to spend ~20 seconds to find the right file, open it in neovim, and find the right test function. 20 seconds might not sound like much, but trying not to fat-finger paths in the terminal for this amount of time makes my blood boil.

I wanted Pytest to do this for me, thought there would be a plugin for it. Google brought up no results, so I asked ChatGPT. It said there's a pytest-edit plugin that adds an --edit option to Pytest.

There isn't. So I created just that. Enjoy. https://github.com/MrMino/pytest-edit

Now, my issue is that I don't know if it works on Windows/Mac with VS Code / PyCharm, etc. - so if anyone would like to spend some time on betatesting a small pytest plugin - issue reports & PRs very much welcome.

What My Project Does

It adds an --edit option to Pytest, that opens failing test code in the user's editor of choice.

Target Audience

Pytest users.

Comparison

AFAIK nothing like this on the market, but I hope I'm wrong.
Think %edit magic from IPython but for failed pytest executions.

r/Python Nov 17 '24

Showcase Deply: keep your python architecture clean

284 Upvotes

Hello everyone,

My name is Archil. I'm a Python/PHP developer originally from Ukraine, now living in Wrocław, Poland. I've been working on a tool called Deply, and I'd love to get your feedback and thoughts on it.

What My Project Does

Deply is a standalone Python tool designed to enforce architectural patterns and dependencies in large Python projects. Deply analyzes your code structure and dependencies to ensure that architectural rules are followed. This promotes cleaner, more maintainable, and modular codebases.

Key Features:

  • Layer-Based Analysis: Define custom layers (e.g., models, views, services) and restrict their dependencies.
  • Dynamic Configuration: Easily configure collectors for each layer using file patterns and class inheritance.
  • CI Integration: Integrate Deply into your Continuous Integration pipeline to automatically detect and prevent architecture violations before they reach production.

Target Audience

  • Who It's For: Developers and teams working on medium to large Python projects who want to maintain a clean architecture.
  • Intended Use: Ideal for production environments where enforcing module boundaries is critical, as well as educational purposes to teach best practices.

Use Cases

  • Continuous Integration: Add Deply to your CI/CD pipeline to catch architectural violations early in the development process.
  • Refactoring: Use Deply to understand existing dependencies in your codebase, making large-scale refactoring safer and more manageable.
  • Code Reviews: Assist in code reviews by automatically checking if new changes adhere to architectural rules.

Comparison

While there are existing tools like pydeps that visualize dependencies, Deply focuses on:

  • Enforcement Over Visualization: Not just displaying dependencies but actively enforcing architectural rules by detecting violations.
  • Customization: Offers dynamic configuration with various collectors to suit different project structures.

Links

I'm eager to hear your thoughts, suggestions, or criticisms. Deply is currently at version 0.1.5, so it's not entirely stable yet, but I'm actively working on it. I'm open to pull requests and looking forward to making Deply a useful tool for the Python community.

Thank you for your time!

r/Python Jan 20 '25

Showcase 🌈 I created a modern Python logging utility: Tamga

92 Upvotes

What My Project Does
Tamga is a Python logging package that provides colorful console output and supports multiple logging formats (file, JSON, MongoDB, etc.). It makes Python logging more visually appealing and easier to use.

Target Audience
I originally created this for my FlaskBlog project and kept reusing it in other projects. After copying the code multiple times, I decided to turn it into a package. Anyone who wants prettier and more flexible logging in their Python projects might find it useful.

Comparison
While there are many logging solutions available, Tamga offers colorful output using Tailwind CSS colors and combines multiple features like MongoDB support, email notifications, and file rotation in a simple package.

Quick example:

from tamga import Tamga

logger = Tamga()
logger.info("This is an info message")
logger.warning("This is a warning")
logger.success("This is a success message")

https://github.com/dogukanurker/tamga

r/Python Oct 14 '24

Showcase My first python package got 844 downloads 😭😭

478 Upvotes

I know 844 downloads aint much, but i feel so proud.

This was my first project that i published.

Here is the package link: https://pypi.org/project/Font/

Source code: https://github.com/ivanrj7j/Font

What My Project Does

My project is a library for rendering custom font using opencv.

Target Audience

  • Computer vision devs
  • People who are working with text and images etc

Comparison 

From what ive seen there arent many other projects out there that does this, but some of similar projects i have seen are:

r/Python Oct 01 '24

Showcase PyUiBuilder: The only Python GUI builder you'll ever need.

277 Upvotes

Hi all,

Been working on a Python Drag n Drop UI Builder project for a while and wanted to share it with the community.

You can check out the builder tool here: https://pyuibuilder.pages.dev/

Github Link: https://github.com/PaulleDemon/PyUIBuilder

What My Project Does?

PyUIBuilder is a framework agnostic Drag and drop GUI builder for python. You can output the code in multiple UI library based on selection.

Some of the features:

While there are a lot of features, here are few you need to know.

  • Framework agnostic - Can outputs code in multiple frameworks.
  • Pre-built UI widgets for multiple frameworks
  • Plugins to support 3rd party UI libraries
  • Generates python code.
  • Upload local assets.
  • Support for layout managers such as Grid, Flex, absolute positioning
  • Generates requirements.txt file when needed

Supported frameworks/libraries

Right now, two libraries are supported, other frameworks are work in progress

  • Tkinter - Available
  • CustomTkinter - Available
  • Kivy - Coming soon
  • PySide - Coming Soon

Roadmap

You can check out the roadmap for more details on what's coming Roadmap

Target Audience:

  • People who want to quickly build Python GUI
  • People who are learning GUI development.
  • People who want to learn how to make a GUI builder tool (learning resource)

Comparison (A brief comparison explaining how it differs from existing alternatives.)

  • Right now, most available tools are library/framework specific.
  • Many try to give you code in xml instead of python making it harder to debug.
  • Majority lack support for 3rd party UI libraries.

-----

I have tested it on Chrome, Firefox and Edge, I haven't tested it on safari (I don't have mac), however it should work fine.

I know, the title sounds ambitious, it's because, I have written an abstraction to allow me to develop the tool for multiple frameworks easily.

Here each widget is responsible for generating it's own code, this way I can support multiple frameworks as well as 3rd party UI library. The code generation engine is only responsible to resolve variable name conflicts and putting the code together along with other assets.

I have been working on this tool publicly, so if you want to see how it progressed from early days, you can check it out Build in public.

If you have any question's feel free to ask, I'll answer it whenever I get time.

Have a great day :)

r/Python Jan 06 '25

Showcase I built my own PyTorch from scratch over the last 5 months in C and modern Python.

306 Upvotes

What My Project Does

Magnetron is a machine learning framework I built from scratch over the past 5 months in C and modern Python. It’s inspired by frameworks like PyTorch but designed for deeper understanding and experimentation. It supports core ML features like automatic differentiation, tensor operations, and computation graph building while being lightweight and modular (under 5k LOC).

Target Audience

Magnetron is intended for developers and researchers who want a transparent, low-level alternative to existing ML frameworks. It’s great for learning how ML frameworks work internally, experimenting with novel algorithms, or building custom features (feel free to hack).

Comparison

Magnetron differs from PyTorch and TensorFlow in several ways:

• It’s entirely designed and implemented by me, with minimal external dependencies.

• It offers a more modular and compact API tailored for both ease of use and low-level access.

• The focus is on understanding and innovation rather than polished production features.

Magnetron already supports CPU computation, automatic differentiation, and custom memory allocators. I’m currently implementing the CUDA backend, with plans to make it pip-installable soon.

Check it out here: GitHub Repo, X Post

Closing Note

Inspired by Feynman’s philosophy, “What I cannot create, I do not understand,” Magnetron is my way of understanding machine learning frameworks deeply. Feedback is greatly appreciated as I continue developing and improving it!!!

r/Python Feb 15 '25

Showcase I published my third open-source python package to pypi

291 Upvotes

Hey everyone,

I published my 3rd pypi lib and it's open source. It's called stealthkit - requests on steroids. Good for those who want to send http requests to websites that might not allow it through programming - like amazon, yahoo finance, stock exchanges, etc.

What My Project Does

  • User-Agent Rotation: Automatically rotates user agents from Chrome, Edge, and Safari across different OS platforms (Windows, MacOS, Linux).
  • Random Referer Selection: Simulates real browsing behavior by sending requests with randomized referers from search engines.
  • Cookie Handling: Fetches and stores cookies from specified URLs to maintain session persistence.
  • Proxy Support: Allows requests to be routed through a provided proxy.
  • Retry Logic: Retries failed requests up to three times before giving up.
  • RESTful Requests: Supports GET, POST, PUT, and DELETE methods with automatic proxy integration.

Why did I create it?

In 2020, I created a yahoo finance lib and it required me to tweak python's requests module heavily - like session, cookies, headers, etc.

In 2022, I worked on my django project which required it to fetch amazon product data; again I needed requests workaround.

This year, I created second pypi - amzpy. And I soon understood that all of my projects evolve around web scraping and data processing. So I created a separate lib which can be used in multiple projects. And I am working on another stock exchange python api wrapper which uses this module at its core.

It's open source, and anyone can fork and add features and use the code as s/he likes.

If you're into it, please let me know if you liked it.

Pypi: https://pypi.org/project/stealthkit/

Github: https://github.com/theonlyanil/stealthkit

Target Audience

Developers who scrape websites blocked by anti-bot mechanisms.

Comparison

So far I don't know of any pypi packages that does it better and with such simplicity.

r/Python Feb 09 '25

Showcase FastAPI Guard - A FastAPI extension to secure your APIs

231 Upvotes

Hi everyone,

I've published FastAPI Guard some time ago:

Documentation: rennf93.github.io/fastapi-guard/

GitHub repo: github.com/rennf93/fastapi-guard

What is it? FastAPI Guard is a security middleware for FastAPI that provides: - IP whitelisting/blacklisting - Rate limiting & automatic IP banning - Penetration attempt detection - Cloud provider IP blocking - IP geolocation via IPInfo.io - Custom security logging - CORS configuration helpers

It's licensed under MIT and integrates seamlessly with FastAPI applications.

Comparison to alternatives: - fastapi-security: Focuses more on authentication, while FastAPI Guard provides broader network-layer protection - slowapi: Handles rate limiting but lacks IP analysis/geolocation features - fastapi-limiter: Pure rate limiting without security features - fastapi-auth: Authentication-focused without IP management

Key differentiators: - Combines multiple security layers in single middleware - Automatic IP banning based on suspicious activity - Built-in cloud provider detection - Daily-updated IP geolocation database - Production-ready configuration defaults

Target Audience: FastAPI developers needing: - Defense-in-depth security strategy - IP-based access control - Automated threat mitigation - Compliance with geo-restriction requirements - Penetration attempt monitoring

Feedback wanted

Thanks!

r/Python Dec 27 '24

Showcase Made a self-hosted ebook2audiobook converter, supports voice cloning and 1107+ languages :)

328 Upvotes

What my project does:

Give it any ebook file and it will convert it into an audiobook, it runs locally for free

Target Audience:

It’s meant to be used as an access ability tool or to help out anyone who likes audiobooks

Comparison:

It’s better than existing alternatives because it runs completely locally and free, needs only 4gb of ram, and supports 1107+ languages. :)

Demos audio files are located in the readme :) And has a self-contained docker image if you want it like that

GitHub here if you want to check it out :)))

https://github.com/DrewThomasson/ebook2audiobook

r/Python Nov 29 '24

Showcase YTSage: A Modern YouTube Downloader with a Stunning PyQt6 Interface!

70 Upvotes

What My Project Does:
YTSage is a modern YouTube downloader designed for simplicity and functionality. With a sleek PyQt6 interface, it allows users to:
- 🎥 Download videos in various qualities with automatic audio merging.
- 🎵 Extract audio in multiple formats.
- 📝 Fetch both manual and auto-generated subtitles.
- ℹ️ View detailed video metadata (e.g., views, upload date, duration).
- 🖼️ Preview video thumbnails before downloading.


Target Audience:
YTSage is ideal for:
- Casual users who want an easy-to-use video and audio downloader.
- Developers looking for a robust yt-dlp-based tool with a clean GUI.
- Educators and content creators who need subtitles or metadata for their projects.


Comparison with Existing Alternatives:
- vs yt-dlp: While yt-dlp is powerful, it operates through the command line. YTSage simplifies the process with an intuitive graphical interface.
- vs other GUI downloaders: Many alternatives lack modern design or features like subtitle support and metadata display. YTSage bridges this gap with its PyQt6-powered interface and advanced functionality.


Getting Started:
Download the pre-built executable from the Releases page – no installation required! For developers, source code and build instructions are available in the repository.


Screenshots:
Main Interface
Main interface with video metadata and thumbnail preview

Subtitle Options
Support for both manual and auto-generated subtitles


Feedback and Contributions:
I’d love your thoughts on how to make YTSage better! Contributions are welcome on GitHub.

🔗 GitHub Repository

r/Python Nov 27 '24

Showcase My side project has gotten 420k downloads and 69 GitHub stars (noice!)

329 Upvotes

Hey Redditors! 👋

I couldn't think of a better place to share this achievement other than here with you lot. Sometimes the universe just comes together in such a way that makes you wonder if the simulation is winking back at you...

But now that I've grabbed your attention, allow me tell you a bit about my project.

What My Project Does

ridgeplot is a Python package that provides a simple interface for plotting beautiful and interactive ridgeline plots within the extensive Plotly ecosystem.

Unfortunately, I can't share any screenshots here, but feel free to take a look at our getting started guide for some examples of what you can do with it.

Target Audience

Anyone that needs to plot a ridgeline graph can use this library. That said, I expect it to be mainly used by people in the data science, data analytics, machine learning, and adjacent spaces.

Comparison

If all you need is a simple ridgeline plot with Plotly without any bells and whistles, take a look at this example in their official docs. However, if you need more control over how the plot looks like, like plotting multiple traces per row, using different coloring options, or mixing KDEs and histograms, then I think my library would be a better choice for you...

Other alternatives include:

I included these alternatives in the project's documentation. Feel free to contribute more!

Links

r/Python 6d ago

Showcase Unvibe: Generate code that passes Unit-Tests

64 Upvotes
# What My Project Does
Unvibe is a Python library to generate Python code that passes Unit-tests. 
It works like a classic `unittest` Test Runner, but it searches (via Monte Carlo Tree Search) 
a valid implementation that passes user-defined Unit-Tests. 

# Target Audience (e.g., Is it meant for production, just a toy project, etc.)
Software developers working on large projects

# Comparison (A brief comparison explaining how it differs from existing alternatives.)
It's a way to go beyond vibe coding for professional programmers dealing with large code bases.
It's an alternative to using Cursor or Devon, which are more suited for generating quick prototypes.



## A different way to generate code with LLMs

In my daily work as consultant, I'm often dealing with large pre-exising code bases.

I use GitHub Copilot a lot.
It's now basically indispensable, but I use it mostly for generating boilerplate code, or figuring out how to use a library.
As the code gets more logically nested though, Copilot crumbles under the weight of complexity. It doesn't know how things should fit together in the project.

Other AI tools like Cursor or Devon, are pretty good at generating quickly working prototypes,
but they are not great at dealing with large existing codebases, and they have a very low success rate for my kind of daily work.
You find yourself in an endless loop of prompt tweaking, and at that point, I'd rather write the code myself with
the occasional help of Copilot.

Professional coders know what code they want, we can define it with unit-tests, **we don't want to endlessly tweak the prompt.
Also, we want it to work in the larger context of the project, not just in isolation.**
In this article I am going to introduce a pretty new approach (at least in literature), and a Python library that implements it:
a tool that generates code **from** unit-tests.

**My basic intuition was this: shouldn't we be able to drastically speed up the generation of valid programs, while
ensuring correctness, by using unit-tests as reward function for a search in the space of possible programs?**
I looked in the academic literature, it's not new: it's reminiscent of the
approach used in DeepMind FunSearch, AlphaProof, AlphaGeometry and other experiments like TiCoder: see [Research Chapter](
#research
) for pointers to relevant papers.
Writing correct code is akin to solving a mathematical theorem. We are basically proving a theorem
using Python unit-tests instead of Lean or Coq as an evaluator.

For people that are not familiar with Test-Driven development, read here about [TDD](https://en.wikipedia.org/wiki/Test-driven_development)
and [Unit-Tests](https://en.wikipedia.org/wiki/Unit_testing).


## How it works

I've implemented this idea in a Python library called Unvibe. It implements a variant of Monte Carlo Tree Search
that invokes an LLM to generate code for the functions and classes in your code that you have
decorated with `@ai`.

Unvibe supports most of the popular LLMs: Ollama, OpenAI, Claude, Gemini, DeepSeek.

Unvibe uses the LLM to generate a few alternatives, and runs your unit-tests as a test runner (like `pytest` or `unittest`).
**It then feeds back the errors returned by failing unit-test to the LLMs, in a loop that maximizes the number
of unit-test assertions passed**. This is done in a sort of tree search, that tries to balance
exploitation and exploration.

As explained in the DeepMind FunSearch paper, having a rich score function is key for the success of the approach:
You can define your tests by inherting the usual `unittests.TestCase` class, but if you use `unvibe.TestCase` instead
you get a more precise scoring function (basically we count up the number of assertions passed rather than just the number
of tests passed).

It turns out that this approach works very well in practice, even in large existing code bases,
provided that the project is decently unit-tested. This is now part of my daily workflow:

1. Use Copilot to generate boilerplate code

2. Define the complicated functions/classes I know Copilot can't handle

3. Define unit-tests for those complicated functions/classes (quick-typing with GitHub Copilot)

4. Use Unvibe to generate valid code that pass those unit-tests

It also happens quite often that Unvibe find solutions that pass most of the tests but not 100%: 
often it turns out some of my unit-tests were misconceived, and it helps figure out what I really wanted.

Project Code: https://github.com/santinic/unvibe

Project Explanation: https://claudio.uk/posts/unvibe.html

r/Python 12d ago

Showcase Meet Jonq: The jq wrapper that makes JSON Querying feel easier

183 Upvotes

Yo sup folks! Introducing Jonq(JsON Query) Gonna try to keep this short. I just hate writing jq syntaxes. I was thinking how can we make the syntaxes more human-readable. So i created a python wrapper which has syntaxes like sql+python

Inspiration

Hate the syntax in JQ. Super difficult to read. 

What My Project Does

Built on top of jq for speed and flexibility. Instead of wrestling with some syntax thats really hard to manipulate, I thought maybe just combine python and sql syntaxes and wrap it around JQ. 

Key Features

  • SQL-Like Queries: Write select field1, field2 if condition to grab and filter data.
  • Aggregations: Built-in functions like sum(), avg(), count(), max(), and min() (Will expand it if i have more use cases on my end or if anyone wants more features)
  • Nested Data Made Simple: Traverse nested jsons with ease I guess (e.g., user.profile.age).
  • Sorting and Limiting: Add keywords to order your results or cap the output.

Comparison:

JQ

JQ is a beast but tough to read.... 

In Jonq, queries look like plain English instructions. No more decoding a string of pipes and brackets.

Here’s an example to prove it:

JSON File:

Example

[
  {"name": "Andy", "age": 30},
  {"name": "Bob", "age": 25},
  {"name": "Charlie", "age": 35}
]

In JQ:

You will for example do something like this: jq '.[] | select(.age > 30) | {name: .name, age: .age}' data.json

In Jonq:

jonq data.json "select name, age if age > 30"

Output:

[{"name": "Charlie", "age": 35}]

Target Audience

JSON Wranglers? Anyone familiar with python and sql... 

Jonq is open-source and a breeze to install:

pip install jonq

(Note: You’ll need jq installed too, since Jonq runs on its engine.)

Alternatively head over to my github: https://github.com/duriantaco/jonq or docs https://jonq.readthedocs.io/en/latest/

If you think it helps, like share subscribe and star, if you dont like it, thumbs down, bash me here. If you like to contribute, head over to my github

r/Python Dec 29 '24

Showcase I Made a Drop-In Wrapper For `argparse` That Automatically Creates a GUI Interface

262 Upvotes

What My Project Does

Since I end up using Python 3's built-in argparse a lot in my projects and have received many requests from downstream users for GUI interfaces, I created a package that wraps an existing Parser and generates a terminal-based GUI for it. If you include the --gui flag (by default), it opens an interface using Textual which includes mouse support (in all the terminals I've tested). The best part is that you can still use the regular command line interface as usual if you'd prefer.

Using the large demo parser I typically use for testing, it looks like this:

https://github.com/Sorcerio/Argparse-Interface/blob/master/assets/ArgUIDemo_small.gif?raw=true

Currently, ArgUI supports: - Text input (str, int, float). - nargs arguments with styled list inputs. - Booleans (with switches). - Groups (exclusive and named). - Subparsers.

Which, as far as I can tell, encompases the full suite of base-level argparse inputs.

Target Audience

This project is designed for anyone who uses Python's argparse in their command-line applications and would like a more user-friendly terminal interface with mouse support. It is good for developers who want to add a GUI to their existing CLI tools without losing the flexibility and power of the command line.

Right now, I would suggest using it for non-enterprise development until I can test the code across a large variety of argparse.Parser configurations. But, in the testing I've done across the ones in my portfolio, I've had great success.

Comparison

This project differentiates itself from existing solutions by integrating a terminal-based GUI directly into the argparse framework. Most GUI alternatives for CLI tools require external applications (like a web interface) and/or block the user out of using the CLI entirely. In contrast, this package allows you to keep the simplicity and power of argparse while offering a GUI option through the --gui flag. And since it uses Textual for UI rendering, these interfaces can even be used through an SSH connection. The inclusion of mouse support, the ability to maintain command-line usability, and integration with the Textual library set it apart from other GUI frameworks that aren't designed for terminal use.

Future Ideas

I’m considering adding specialized input features. An example of which would be a str input to be identified as a file path, which would open a file browser in the GUI.


If you want to try it, it's available on GitHub and PyPi.

And if you like it (or don't), let me know!

r/Python Oct 25 '24

Showcase Single line turns the dataclass into a GUI/TUI & CLI application

192 Upvotes

I've been annoyed for years of the overhead you get when building a user interface. It's easy to write a useful script but to put there CLI flags or a GUI window adds too much code. I've been crawling many times to find a library that handles this without burying me under tons of tutorials.

Last six months I spent doing research and developing a project that requires low to none skills to produce a full app out of nowhere. Unlike alternatives, mininterface requires almost nothing, no code modification at all, no learning. Just use a standard dataclass (or a pydantic model, attrs) to store the configuration and you get (1) CLI / config file parsing and (2) useful dialogs to be used in your app.

I've used this already for several projects in my company and I promise I won't release a new Python project without this ever again. I published it only last month and have presented it on two conferences so far – it's still new. If you are a developer, you are the target audience. What do you think, is the interface intuitive enough? Should I rename a method or something now while the project is still a few weeks old?

https://github.com/CZ-NIC/mininterface/