Why Social Media Scrapers Keep Breaking: A Checklist

Social media scrapers keep breaking when the assumptions they depend on stop matching the source. Your collector can lose access, encounter different fields or pagination, or extract the wrong part of a page. A platform update is one possible cause; an expired token or exhausted quota needs a different response.

This guide is for agencies and founders responsible for social-data inputs to research or reporting. Start with the failure matrix, record the source contract, then test the missing-field incident. For the broader effect on creator tools, see the existing social media API changes overview.

For a research task using your own stored content, check workspace evidence.

Identify What You Are Collecting From

Name the access method before diagnosing it:

• Documented platform API: record the endpoint, API version where applicable, granted scopes and the response fields you use.
• HTML extraction: record the page, selector or extraction rule, and whether the saved response actually contains the required content.
• Undocumented/internal endpoint: record that dependency explicitly. A wrapper around a website’s internal request is not evidence of a supported public API contract.
• Stored dataset or connector: record which upstream source supplies it, when the observation was made and what the connector exposes.

These paths fail differently. If a selector expects a field in a page and that field is absent from the saved HTML, changing an API token will not fix that extraction rule. If a documented API denies permission, changing the selector will not supply permission.

Define the downstream job too. “Read the captions of our last five videos” needs different evidence from “export every public video in a market with continuously current metrics.” This distinction determines whether you need to maintain collection, request a permitted source, or read content you already hold.

Diagnose the Failure Before Changing the Collector

Capture a redacted failed response and a known successful one where available. Keep the status, error reason, request time and selected fields; remove tokens and private payloads before sharing the incident.

Use platform error documentation for the diagnosis. YouTube’s API error reference distinguishes permission failures, exhausted quota and invalid page tokens. The same HTTP status can require different actions: its 403 responses include both access and quota reasons. Read the reason before retrying.

TikTok’s rate-limit documentation describes endpoint-specific limits and the 429 rate_limit_exceeded response. Its token-management documentation describes expiry, granted scopes and refresh behavior. A valid-looking token string is not evidence that the required access is still available.

Check Meaning as Well as Shape

A field can still exist while its meaning changes. YouTube’s revision history records the March 31, 2025 change to Shorts view counting, including the relevant Data API view-count fields. A schema check alone cannot tell you whether two periods use the same metric definition.

Put the definition and date beside the value. Separate a collection failure from a change in measurement. Do not describe either as an audience decline until you have comparable observations.

Write a Source Contract You Can Check

A source contract is a short record of what your workflow expects, how you check it and who resolves a mismatch. It is your operating document, not a promise from the platform.

Copy this worksheet for each source feeding a decision:

Job the data supports:
Source / endpoint / page:
Access method: documented API / HTML / internal endpoint / stored dataset
Account or resource scope:
Required access and where it is confirmed:
Fields required for this job:
Metric definitions and units:
API/schema version or extraction rule:
Source observation timestamp, or unavailable:
Our collection timestamp:
Expected update cadence and stale-data rule:
Pagination / record identifier rule:
Missing value rule: unavailable is not zero
Validation: required fields, types, IDs and comparable definitions
Failure evidence: status, reason, redacted response and request time
Downstream decision paused when evidence fails:
Incident owner:
Next review date:

Keep publication time and observation time separate. A video published yesterday can have metrics collected today; the video’s publication date does not tell you when those metrics were observed. If your source supplies no observation timestamp, record that limitation.

Set the stale-data rule around the actual job. A historical content review can use an explicitly dated snapshot. A decision that requires current figures needs a source whose observation time and update cadence you can establish. Do not invent a universal freshness threshold for every workflow.

For shared client work, keep this record with your management operating guides, source evidence and incident owner. A successful scheduled run should pass the data checks as well as complete its request.

Worked Incident: Unavailable Is Different From Zero

Hypothetical diagnostic fixture: an agency reviews three of its own videos. All records and numbers below are invented to demonstrate the check; they are not Mysocial customer results or platform benchmarks. Assume the first two observations use the same view definition and review window.

A mapping that replaces every missing field with 0 would make Video C look like Video B. You would lose the distinction between “the source reports zero” and “the source did not supply this field.”

The observed subset sums to 120 + 0 = 120 views across two records. Required-field coverage is 2 of 3 records. The three-record total is unknown. Dividing 120 by three and reporting an average of 40 would include an unsupported zero for Video C; it would not describe three measured observations.

Investigate Video C before publishing a complete comparison:

1. Preserve the redacted response and field mapping.
2. Check whether the field is absent in the source or lost in your transformation.
3. Review access and dated schema/definition documentation for that source.
4. Record the cause only when the evidence establishes it.
5. Rerun the bounded check after a supported correction, then update the incident record.

Do not overwrite a prior dated observation with a fabricated current zero. If you retain the prior value, show its original observation time and state that no comparable current figure is available.

Decide Whether You Need New Collection

Repair or replace a collection path when your job requires data you do not have, and confirm that the alternative source supplies the required scope and fields. Changing a provider name does not by itself establish completeness, freshness or equivalent metric definitions.

For a narrower research task, you can sometimes work from existing content instead. If the question is “what did our own videos say, and what evidence is available for comparing them?”, begin with the content your workspace already holds. Keep missing transcripts, metrics and observation times visible in the result.

Our social media MCP server comparison separates reading your content, publishing, reporting and outside research. Choose the job before choosing the connection. When you prepare the evidence table, the guide to formatting tables with ChatGPT covers readable columns; the source contract above determines what those columns can truthfully contain.

Use Stored Content for a Bounded Research Task

Connect Mysocial to your assistant when you want it to read content already available in your selected workspace. The stored-content path uses search_content to find your own posts and get_content to open the returned references. Access depends on your plan and workspace, and a read can return incomplete or unavailable evidence.

Use the prompt below only for that narrower job. Replace the topic and period with your actual brief.

Check the evidence already in your workspace

@Mysocial MCP

Edit your prompt Use Mysocial only to read content already available in the workspace I selected. Use search_content and get_content to review up to five of my own posts about [topic] published during [period]. Return each source reference or permalink, publication date, relevant available content and available metrics with their definitions or limits. Keep publication time separate from metric observation time. Mark missing fields, transcripts, observation timestamps and freshness unknown when they are not supplied; never convert an unavailable value to zero. State which comparisons the available evidence supports. If access, plan limits or missing content block the task, explain the specific gap and stop. Do not start external research, refresh collection, trigger analysis, save anything or send messages.

Review and send in your AI app.

Setup & access

Connect and select Mysocial before sending. Connect Mysocial →

A connected Mysocial workspace and the plan access required for its stored content. This task reads what is already available; it does not repair a scraper, guarantee current observations or recover missing history.

Result: A small evidence table with source references, available content and explicit gaps for your review.

The Run buttons open your chosen AI app. Connect and select Mysocial there, then review before sending; copy and paste if needed. If copying is unavailable, select the visible prompt and copy it manually.

Finish by naming the decision the evidence supports and the input still missing. Your next action should follow the diagnosis: resolve access, adjust request load, correct a mapping, annotate a metric change or obtain a suitable dated source. Keep the source contract beside that decision so the next person can see what was checked.