robots.txt
See which AI crawlers other sites block →# -----------------------------------------------
# The Lever: robots.txt | Updated: September 2026
# -----------------------------------------------
# --- All crawlers: block Ghost admin/API paths ---
User-agent: *
Disallow: /ghost/
Disallow: /.ghost/
Disallow: /account/
# --- AI answer/retrieval bots are allowed by default ---
# OAI-SearchBot, Claude-SearchBot, Claude-User, PerplexityBot, ChatGPT-User,
# Perplexity-User, Applebot, Bingbot, Googlebot, DuckDuckBot cite sources and
# send referral traffic. FacebookExternalHit powers link previews on Facebook,
# Instagram, Messenger, and WhatsApp. Allowed here. Meta's training crawler
# is Meta-ExternalAgent, blocked below.
# OpenAI: training
User-agent: GPTBot
Disallow: /
# Anthropic: training
User-agent: ClaudeBot
Disallow: /
User-agent: anthropic-ai
Disallow: /
# Google: Gemini training only (does not affect Search rankings or AI Overviews)
User-agent: Google-Extended
Disallow: /
# Meta: training and product data collection
User-agent: Meta-ExternalAgent
Disallow: /
# Apple: Apple Intelligence training
User-agent: Applebot-Extended
Disallow: /
# Cohere: model training
User-agent: cohere-ai
Disallow: /
User-agent: cohere-training-data-crawler
Disallow: /
# ByteDance/TikTok: model training
User-agent: Bytespider
Disallow: /
# Common Crawl: open training dataset
User-agent: CCBot
Disallow: /
# Huawei: AI training
User-agent: PetalBot
Disallow: /
# Diffbot: AI dataset scraping
User-agent: Diffbot
Disallow: /
# Omgili/WebzIO: content aggregation and training
User-agent: Omgilibot
Disallow: /
User-agent: Webzio-Extended
Disallow: /
# Image scraping for AI training
User-agent: img2dataset
Disallow: /
# Other training bots
User-agent: AI2Bot
Disallow: /
User-agent: PanguBot
Disallow: /
# --- Sitemap ---
Sitemap: https://www.levernews.com/sitemap.xml
llms.txt
Compare with other sites' llms.txt files →# The Lever
> A reader-supported investigative news organization holding power accountable.
Public Ghost content for AI and LLM tooling. Use `/llms-full.txt` for consolidated page and post context.
Append `.md` to any post or page URL to get the content in Markdown (for example, `/example-post.md`).
## Pages
- [About The Daily Poster](https://www.levernews.com/about-old.md) - Welcome to The Daily Poster — a reader-supported investigative journalism project that covers politics, business and corporate power. The project is led by David Sirota, an award-winning journalist, Guardian columnist, veteran poster, and former presidential campaign speechwriter for Bernie Sanders…
- [Informed Citizens Make Better Choices](https://www.levernews.com/about.md) - The Lever is a reader-supported investigative news outlet that holds accountable the people and corporations manipulating the levers of power.
- [Access The Citizens' Guide To Holding The Powerful Accountable](https://www.levernews.com/citizens-guide-access.md)
- [The Citizens' Guide To Holding The Powerful Accountable](https://www.levernews.com/citizens-guide-to-holding-the-powerful-accountable.md) - The Daily Poster's new publication for subscribers gives you our reporting techniques to follow the money and scrutinize power.
- [The Citizens' Guide To Holding The Powerful Accountable](https://www.levernews.com/citizens-guide.md) - Your inside guide to all of the tools we use to scrutinize power.
- [Contact Us](https://www.levernews.com/contact.md) - Get in touch with The Lever staff regarding any tips, questions, suggestions, subscriptions, etc.
- [FAQ](https://www.levernews.com/faq.md) - Here are answers to common questions that you may have about our website:
- [Gift A Subscription](https://www.levernews.com/gift.md) - Now you can give a full subscription to someone you love (or hate).
- [How To Create, Manage, Upgrade, Or Cancel Your Subscription](https://www.levernews.com/how-to-manage-your-account.md) - Click the links for answers to these questions:
- [Opportunities](https://www.levernews.com/jobs.md) - The following positions are currently open:
- [Support The Lever](https://www.levernews.com/membership.md) - CHOOSE YOUR SUBSCRIPTION
- [How To Pitch Us](https://www.levernews.com/pitch.md) - Have a killer story idea? Send it our way.
- [Podcasts From The Lever](https://www.levernews.com/podcasts.md) - The Lever’s podcast network includes Lever Time, The Audit, Movies vs. Capitalism, Lever Premium Podcasts, and more. Our shows combine hard-hitting news with insightful analysis.
- [Politics](https://www.levernews.com/politics.md) - Breaking news and analysis on U.S. politics, including the latest coverage of the White House, Congress, and the Supreme Court.
- [Privacy Policy](https://www.levernews.com/privacy.md) - Find answers about our privacy policy here.
- [Subscribe to The Lever](https://www.levernews.com/subscribe.md) - Supporting subscriptions grant you complete access to our entire website and all content for just $10 per month, or $100 per year.
- [Thank You For Becoming A Supporting Subscriber!](https://www.levernews.com/thank-you-for-becoming-a-paid-subscriber.md) - You are now a part of holding the powerful accountable.
- [America is in crisis. Here’s how you can help…](https://www.levernews.com/thanks-for-signing-up.md) - Thanks for becoming part of The Lever’s community, which is over 200,000 concerned citizens strong — and growing every day.
- [Update Payment](https://www.levernews.com/update-payment.md) - If you are having trouble updating your payment method via this form (which can happen if the name on your credit card doesn't match the name on your account), you can click here [https://www.levernews.com/account/]instead. If you click on the above link, you will be directed to a window detailing…
- [Daily Poster Special Discount For Westword Readers](https://www.levernews.com/westword.md) - Click the button below for a special 15 percent discount to an annual subscription to The Daily Poster, designed exclusively for Westword readers! Click Here For The Westword Discount [https://dailyposter.outpost.pub/public/promo-subscription/b8107b5a2c6bf0af473#/]
## Posts
- [Lever Weekly: The Wave Is Over. The Fight Isn’t.](https://www.levernews.com/lever-weekly-the-wave-is-over-the-fight-isnt.md) - Billionaire backfires, Democratic power struggles, and more from The Lever this week.
- [Corporate Democrats Just Admitted Bernie Was Right](https://www.levernews.com/corporate-democrats-just-admitted-bernie-was-right.md) - After declaring war on the left, Third Way is celebrating James Talarico — and admitting that Bernie Sanders-style populism is now the political mainstream.
- [You Love To See It: The Other Winners Of Election Night](https://www.levernews.com/you-love-to-see-it-the-other-winners-of-election-night.md) - The rich get their tax bill, the freedom state strikes back, voters cut the corporate money cord, and the AI overlords get evicted.
- [We Exposed Democrats’ Billionaire Backers. Now They’re Blocking Our Reporters.](https://www.levernews.com/we-exposed-democrats-billionaire-backers-now-theyre-blocking-our-reporters.md) - A billionaire-backed group is gathering top Democrats to shape the party's future — while blocking our reporters after we exposed its funding.
- [El-Sayed: Fire The Parliamentarian If She Blocks Healthcare Reform](https://www.levernews.com/el-sayed-fire-the-parliamentarian-if-she-blocks-healthcare-reform.md) - In an exclusive interview, Michigan’s senator-elect says Democrats should replace the Senate parliamentarian if she uses “arcane” rules to block health care legislation.
- [Don’t F#*k It Up, Democrats: The Battle To Break Big Tech’s Power](https://www.levernews.com/dont-f-k-it-up-democrats-the-battle-to-break-big-techs-power.md) - In Episode 2 of The Lever’s new video series, David Sirota asks Gil Duran and Saikat Chakrabarti whether Democrats will confront the tech broligarchy — or bail it out.
- [Who Got Washed Out By The Blue Tsunami?](https://www.levernews.com/levermind-quiz-who-got-washed-out-by-the-blue-tsunami.md) - Test your knowledge of the blue wave, ballot initiatives, and other issues that really matter.
- [The Blue Wave Might Have Washed Away Progressives’ Leverage](https://www.levernews.com/did-the-blue-wave-wash-away-progressives-leverage.md) - Democrats’ landslide strengthened Hakeem Jeffries’s bid for House speaker, weakening progressive insurgents’ bargaining power even as their Senate allies challenge Chuck Schumer.
- [The Fight Over Dems’ Subpoena List](https://www.levernews.com/the-fight-over-dems-subpoena-list.md) - An 181-page plan urges Democrats to put tech executives, Wall Street landlords, and price gougers under oath — but corporate Democrats are pushing back.
- [Will Dems’ Blowout Sideline Progressives?](https://www.levernews.com/will-dems-blowout-sideline-progressives.md) - Tuesday’s blue wave may secure Hakeem Jeffries’ speakership.
- [Don’t F#*k It Up, Democrats: The Fight For The Party's Future](https://www.levernews.com/dont-f-k-it-up-democrats-the-fight-for-the-partys-future.md) - In the first episode of The Lever’s new video series, David Sirota asks Robert Reich and Patrick Gaspard whether Democrats will now answer to voters — or wealthy donors.
- [Big Tech Lost The Election. Now Comes The Gaslighting.](https://www.levernews.com/big-tech-lost-the-election-now-comes-the-gaslighting.md) - As Democrats prepare to investigate Silicon Valley, its liberal-branded lobby is scrambling to erase voters’ resounding rejection of Big Tech.
- [This Is The Moment I Built The Lever For](https://www.levernews.com/this-is-the-moment-i-built-the-lever-for.md) - The midterm results have proven that anti-corruption populism is the winning formula. Now help us make sure Democrats actually deliver.
- [The Myth Of Red Texas Just Died](https://www.levernews.com/the-myth-of-red-texas-just-died.md) - After generations of corporate capture, Texas’ long-suppressed anti–oligarchy movement returned — and swept Democrats into power.
- [It’s A Bad Day For Dark Money And Data Centers](https://www.levernews.com/its-a-bad-day-for-dark-money-and-data-centers.md) - Deep-red voters overwhelmingly approved a plan to ban corporate political spending — creating a potential blueprint to taking elections back from big money.
- [Democrats Took Congress. Can Populists Make It Count?](https://www.levernews.com/democrats-took-congress-can-populists-make-it-count.md) - Texas flipped. El-Sayed won. Congress changed hands. Let’s unpack.
- [Democrats Won. Let’s Make Sure They Don’t F#*k It Up.](https://www.levernews.com/democrats-won-now-dont-f-k-it-up.md) - The voters delivered a mandate for change. Our explosive new series asks whether Democrats have the guts to deliver.
- [The Left’s Election Day With Hasan Piker](https://www.levernews.com/hasan-pikers-take-on-the-elections.md) - Hasan Piker joins David Sirota to break down the progressive insurgency reshaping the midterms — and what comes after Election Day.
- [Musk’s Half-Billion-Dollar Midterms Bet To Stay Out Of Trouble](https://www.levernews.com/musks-half-billion-dollar-midterms-bet-to-stay-out-of-trouble.md) - The world’s richest man is facing new scrutiny from Congress — and apparently trying to spend his way out of accountability.
- [Will The Blue Wave Come For Schumer?](https://www.levernews.com/would-a-blue-wave-put-schumers-leadership-on-the-line.md) - Election Day is finally here. Progressive victories could spell trouble for Leader Schumer.
- [The Midterms May Be An Anti-Trump Wave. They Could Become Far More.](https://www.levernews.com/the-midterms-may-be-an-anti-trump-wave-they-could-become-far-more.md) - The conditions for real change are finally emerging — if voters treat Election Day as the beginning rather than the end.
- [Big AI’s $140 Million Election Machine Stalls Out](https://www.levernews.com/big-ais-140-million-election-machine-stalls-out.md) - A super PAC network became so politically toxic that it stopped donating in the most competitive Congressional races.
- [El-Sayed Posts Seven-Figure Haul After Stevens’ Betrayal](https://www.levernews.com/el-sayed-posts-seven-figure-haul-after-stevens-betrayal.md) - 29,000 of the donations were from first-time givers. Now that's a backfire.
- [Haley Stevens’ Betrayal Is A Warning: Corporate Dems Are MAGA’s Enablers](https://www.levernews.com/haley-stevens-mike-rogers-endorsement.md) - By endorsing Republican Mike Rogers, Abdul El-Sayed’s defeated primary rival proved that Democrats’ pro-oligarch faction is willing to destroy their own party.
- [Lever Weekly: The Final Countdown](https://www.levernews.com/lever-weekly-the-final-countdown.md) - Wayward donors, progressive revolts, and more from The Lever this week.
- [The Left’s Donors Don’t Hold Grudges. The Establishment’s Do.](https://www.levernews.com/the-lefts-donors-dont-hold-grudges-the-establishments-do.md) - Midterm campaign filings reveal a one-way loyalty test: Progressive donors rally behind Democratic nominees, while establishment money disappears — or moves to Republicans — when those nominees are populists.
- [You Love To See It: No Tricks At The Polls](https://www.levernews.com/you-love-to-see-it-no-tricks-at-the-polls.md) - Democracy establishes a no-ICE zone, the revolution gets a routing number, survivors score an $800 million reckoning, and Meta pays up.
- [Democrats Could Win The Election And Lose The War](https://www.levernews.com/democrats-could-win-election-lose-war.md) - The right spent decades building a political ecosystem. Democrats are still acting as if one election can defeat it.
- [Big Tech’s Democratic Front Group: Investigate Trump, Not Big Tech](https://www.levernews.com/big-tech-to-democrats-investigate-trump-not-us.md) - The liberal-flavored Chamber of Progress is pressing Democrats to scrutinize the White House — but avoid probing tech giants that bankrolled and benefited from Trump's presidency.
- [Who’s Running The Democratic Party?](https://www.levernews.com/levermind-quiz-whos-running-the-democratic-party.md) - Test your knowledge of the Democratic Party, A.I. chatbots, and other issues that really matter.
- [Robert Reich Is Nauseously Optimistic About The Midterms](https://www.levernews.com/robert-reich-is-nauseously-optimistic-about-the-midterms.md) - Some Democrats want to chase “Romney Republicans.” Here’s why Robert Reich thinks that’s a losing bet, which candidates he’s watching, and why Chuck Schumer should go if Democrats win the Senate.
- [Gina Hinojosa Flips Off Corporate Democrats](https://www.levernews.com/gina-hinojosa-explains-the-democratic-divide.md) - The surging Texas gubernatorial candidate is explicitly campaigning against Corporate Democrats’ old formula and its useless labels.
- [Prediction Markets Are Shaping Elections — And Getting Them Wrong](https://www.levernews.com/prediction-markets-are-shaping-elections-and-getting-them-wrong.md) - Do Kalshi’s odds favor the establishment?
- [He Helped Make Karen Bass Mayor. Now He’s Funding Her Defeat.](https://www.levernews.com/he-helped-make-karen-bass-mayor-now-hes-funding-her-defeat.md) - A longtime establishment power broker is backing socialist Nithya Raman over Los Angeles Mayor Karen Bass — a sign that some in the donor class are following voters to the left.
- [The Progressive Revolt Stops At Hakeem Jeffries](https://www.levernews.com/the-progressive-revolt-stops-at-hakeem-jeffries.md) - The Democratic left bets it can win more by working with the House Speaker. That could cost them.
- [Big Business Wants To Delete Your Tax Cut](https://www.levernews.com/midterm-madness-big-business-wants-to-delete-your-tax-cut.md) - Corporate interests are throwing a Hail Mary to keep tax reform off the ballot.
- [Corporate Dems Lost Red States. Can Populists Win Them Back?](https://www.levernews.com/corporate-dems-lost-red-states-can-populist-dems-win-them-back.md) - Even in Texas, Florida, and Nebraska, voters are rallying behind populist candidates who want to tax the rich.
- [The Blackmail Warning In Anthropic’s $2 Trillion IPO](https://www.levernews.com/the-blackmail-warning-in-anthropics-2-trillion-ipo.md) - The artificial intelligence company is vying for a historic IPO. Its paperwork reveals key weaknesses.
- [Political Strategists Are Hacking AI To Influence Voters](https://www.levernews.com/political-strategists-are-hacking-ai-to-influence-voters.md) - As voters turn to AI for research, strategists are employing slimy tactics to control the narrative.
- [Lever Live: Behind AIPAC’s $13.1M Money Trail](https://www.levernews.com/lever-live-behind-aipacs-money-trail.md) - David Sirota sits down with Lever reporters Katya Schwenk and Luke Goldstein to dive into their investigation that traces how the Israel lobby is furtively routing millions.
- [Schumer, Jeffries Face A Voter Revolt](https://www.levernews.com/schumer-jeffries-face-a-voter-revolt.md) - Voters in crucial election battlegrounds want to see Hakeem Jeffries and Chuck Schumer ousted from Dem leadership.
- [Lever Weekly: Masked Money, Zombie Votes, And 50 Free Kills](https://www.levernews.com/lever-weekly-masked-money-zombie-votes-and-50-free-kills.md) - Camouflaged election cash, corporate America’s autopilot voters, AI’s body-count threshold, and more from The Lever this week.
- [How Far Will The AI Sector Go?](https://www.levernews.com/how-far-will-the-ai-sector-go.md) - Test your knowledge of AI legislation, dark money groups, and other issues that really matter.
- [AIPAC Is Spending Millions — Just Not Under Its Own Name](https://www.levernews.com/aipac-is-spending-millions-just-not-under-its-own-name.md) - A Lever investigation traces how the Israel lobby is routing millions through groups promoting women, reproductive rights, and Latino candidates — and how AI interests and other power brokers are adopting the strategy.
- [You Love To See It: Cantering For Good News](https://www.levernews.com/you-love-to-see-it-cantering-for-good-news.md) - New York City progresses at a gallop pace, California preserves its Indigenous heritage, and other good news.
- [America’s Most Fascist Candidate, Brought To You By Big Oil](https://www.levernews.com/americas-most-fascist-candidate-brought-to-you-by-big-oil.md) - A Texas candidate trafficking in white-nationalist politics has found powerful allies among the oil interests he promises to deregulate.
- [Angie Nixon's Battle Cry Against Corporate Power](https://www.levernews.com/angie-nixons-battle-cry-against-corporate-power.md) - Lever columnist Morgan Harper sits down with the U.S. Senate candidate from Florida to discuss why calling out corporations is essential to empowering Black Americans.
- [Rahm Emanuel’s Plan Is Exactly How Dems Will Keep Losing](https://www.levernews.com/rahm-emanuels-plan-is-exactly-how-dems-will-keep-losing.md) - Desperate to cast a do-nothing, pro-oligarch agenda as good politics, Democratic elites pretend the key swing vote is country-club Republicans rather than enraged working-class voters.
- [The Blue-State Democrats Fighting To Shield Billionaires From Taxes](https://www.levernews.com/the-blue-state-democrats-fighting-to-shield-billionaires-from-taxes.md) - Class solidarity is one helluva drug among Democratic tycoons-turned-governors.
- [What’s So Radical About Angie Nixon?](https://www.levernews.com/whats-so-radical-about-angie-nixon.md) - The Florida Democrat says she’s no extremist. She’s just running ‘toward the people and what it is that they need.’
- [Ken Salazar Is Campaigning Against A Tax On People Like Ken Salazar](https://www.levernews.com/ken-salazar-is-campaigning-against-a-tax-on-people-like-ken-salazar.md) - Obama’s former interior secretary is using workers and right-wing spin to attack a tax measure that could cost someone earning his former salary an extra $111,000 a year.
- [Kelly Ayotte’s Data Center Reversal Has A Potential Million-Dollar Footnote](https://www.levernews.com/kelly-ayottes-data-center-reversal-has-a-potential-million-dollar-footnote.md) - The incumbent New Hampshire governor helped deregulate data center power providers after serving on Bloom Energy's board — and shares she held could now be worth seven figures.
- [Lindsay Owens: Who Is Your AI Shopping Agent Working For?](https://www.levernews.com/lindsay-owens-ai-surveillance-pricing.md) - Swing voters are furious about something most people can barely name: surveillance pricing.
- [Why Are Democrats The Party Of No?](https://www.levernews.com/medicare-for-none.md) - Weeks before the midterms, the Democratic establishment is insisting their party must stand against the public’s policy demands.
- [The Mystery Voter Guide Using Democrats To Elect Mike Rogers](https://www.levernews.com/the-last-ditch-effort-to-sink-el-sayed.md) - The GOP’s Mike Rogers curiously joins other Democrats on a “pro-democracy” flyer distributed in Michigan.
- [Angie Nixon's Formula For Winning Without Corporate Cash](https://www.levernews.com/angie-nixons-formula-for-winning-without-corporate-cash.md) - U.S. Senate candidate Angie Nixon is challenging Republican Sen. Ashley Moody without money from corporate PACs — or the full blessing of the Democratic establishment
- [AI Industry Leader Wants 50 Free Kills](https://www.levernews.com/ai-leader-wants-50-free-kills.md) - Draft artificial intelligence legislation circulated by Anthropic would permit dozens of projected deaths before triggering government intervention.
- [Lever Weekly: Wall Street Wants You Quiet](https://www.levernews.com/lever-weekly-wall-street-wants-you-quiet.md) - The wealth tax, assault rifles, shareholder antics, and more from The Lever this week.
- [You Love To See It: The Phone Tree Gets the Ax](https://www.levernews.com/you-love-to-see-it-the-phone-tree-gets-the-ax.md) - Indigenous People’s Day raises the alarm, California takes customers off of the endless phone hold — and other wins worth a