A Reader Told Us Our Apple Number Was Wrong. They Were Right.

We published a number. It was wrong. The way it was wrong is worth more than the number was.

The claim

Apple discloses pay on 1 of 4,948 postings through its own careers feed, against 98 of 103 through TheMuse. We presented it as a clean natural experiment: same employer, two channels, and therefore proof that salary transparency is a property of the channel rather than the employer.

It was a good story. It had a named company, a sharp ratio, and a methodological point.

The correction

A reader said, in effect: that cannot be right, I have seen salaries on Apple's site with my own eyes.

California requires pay ranges on job postings. So we took 14 Apple postings with a California location out of our own corpus, opened each URL on jobs.apple.com, and looked.

Ten of the fourteen published an explicit base pay range. One Cupertino role advertised $144,600 to $263,800. Two of eight sampled non-California US roles also had one, both in Seattle, which also has a disclosure law. The non-US ones did not, which is what you would expect.

Apple was disclosing the entire time. The pay sits in a "Pay & Benefits" block that our connector never read. We stored the opening marketing paragraph, a median of 871 characters, and recorded an absence we had manufactured ourselves.

The part that should worry you if you build data products

We had run a control. The original analysis explicitly asked the right question: is the pay merely sitting in the description text, unparsed? It counted dollar amounts in the descriptions of postings with no structured pay figure, and found apple 0 of 4,947.

That control was honest, well-intentioned, and completely worthless. It searched the text we had stored. For Apple that text is an 871-character teaser that could not have contained a pay figure under any circumstances. The control could not fail.

A control that runs against your own transformed copy can only ever confirm your own transformation.

The check that actually caught it was a person remembering what a web page looks like.

It was not one bug

While confirming the Apple mechanism we found a second, unrelated one producing the identical signal.

Amazon publishes pay too, at the very end of a long job description. We cap stored description text at 5,000 characters. One row in our corpus ends mid-sentence on the words "base salary range for this position is listed below" before the cap severs it. The only posting in that sample that came in under the cap, at 3,159 characters, ends with USA, WA, Seattle - 143,700.00 - 194,400.00 USD annually.

Two different mechanisms, one false conclusion: this employer does not disclose pay.

What we changed

The page is corrected rather than deleted, with the original figures quoted so the error stays auditable. Every per-employer claim on it is withdrawn. The thesis is withdrawn, because the evidence for it turned out to measure our own pipeline. Our corpus-wide 33% is now published as a floor on disclosure rather than a rate.

One thing survived and it is the genuinely interesting part. One source in our corpus carries a median description of 453 characters and still reports pay on 80% of its postings, because its feed delivers pay as a structured field rather than as prose. Description length does not predict whether pay survives. Delivery shape does.

The rule we are adopting

Before publishing any claim that something is missing, answer one question first:

Is it absent at the source, or did we fail to preserve it?

Those are different facts. They look identical in a database. We had no way to tell them apart, so we published the flattering interpretation without noticing we had chosen one.

If you read a statistic of the form "X% of job postings don't state Y," the useful question is not whether the sample was big. It is whether anyone checked what the original page said.