How to find HTML elements by text value with BeautifulSoup

by scrapecrow Oct 26, 2022

Using Python and Beautifulsoup we can find any HTML element by partial or exact text value using find / find_all method by passing regular expressions object to the text parameter:

import re
import bs4

soup = bs4.BeautifulSoup('<a>Twitter link</a>')

# case sensitive:
soup.find("a", text=re.compile("Twitter"))  # will find 1st ocurrance 
soup.find_all("a", text=re.compile("Twitter"))  # will find all ocurrances
# case insensitive:
soup.find("a", text=re.compile("twitter", re.I))
soup.find_all("a", text=re.compile("twitter", re.I))

Related Articles

How to Scrape YouTube in 2025

Learn how to scrape YouTube, channel, video, and comment data using Python directly in JSON.

SCRAPEGUIDE
PYTHON
HIDDEN-API
How to Scrape YouTube in 2025

Advanced Proxy Connection Optimization Techniques

Master advanced proxy optimization with TCP connection pooling, TLS fingerprinting, DNS caching, and HTTP/2 multiplexing for maximum performance.

PROXIES
Advanced Proxy Connection Optimization Techniques

Automatic Failover Strategies for Reliable Data Extraction

A deep dive into automatic failover strategies like retries, backoff, and circuit breakers to build resilient and reliable web scrapers that can handle network errors, blocks, and other common failures.

BLOCKING
Automatic Failover Strategies for Reliable Data Extraction

How to Stop Wasting Money on Proxies

Learn actionable techniques to slash proxy spending in web scraping projects without sacrificing reliability or scale.

PROXIES
How to Stop Wasting Money on Proxies

HTTPS vs. SOCKS Proxies

A deep dive into the key differences between HTTPS and SOCKS proxies, helping you choose the right protocol for your web scraping needs.

PROXIES
HTTPS vs. SOCKS Proxies

Optimize Proxy Bandwidth with Image & CSS Stubbing

Reduce proxy costs by 30-50% through intelligent image and CSS stubbing techniques that eliminate unnecessary resource downloads while preserving functionality.

PROXIES
Optimize Proxy Bandwidth with Image & CSS Stubbing