I think it’s more about the web visitor cost. Handling traffic and API calls becomes a financial problem when there are a growing number of companies using bots to scrape data. Larger companies are moving their content behind paywalls, which acts as a bot filter, and have also identified that they can generate a revenue stream from subscriptions and API connections. Old content on the web is not deemed to have much business value, so it’s a decision of either charging for it or scrapping it.