Firestore Missing Fields: Find Schema Gaps and Mixed Types

Find the documents your query is quietly leaving out

Your orders collection has 100 documents, but sorting by createdAt shows only 83. Or price is a number in newer products and a string in records imported months ago. Before you add another index or change your UI, check whether the documents actually share the fields and types your code expects.

This guide explains missing-field query behavior in Firestore Standard/Core, gives you a read-only scan with counts and example document paths, and shows how to inspect the wider schema in Chiselbase. Enterprise Pipeline operations have additional existence functions; those are covered separately below.

Need to understand more than one field? Chiselbase’s free schema map shows observed field presence, types and common values, then generates a starting point for your TypeScript or Zod model. See the schema workflow.

Way 1: understand missing, null and empty values

These three documents represent different states:

[
  { "id": "a", "email": "ada@example.com" },
  { "id": "b", "email": null },
  { "id": "c" }
]

Document b has an email field whose value is null. Document c has no email field. An empty string is another present value; it is not an absent field.

Firebase documents that not-equal and not-in filters exclude absent fields. != also excludes null values. Adding orderBy() requires the ordered field to exist. An index cannot bring absent fields into those query results.

There is no where("email", "exists", false) operator in the Core query API. Setting the comparison value to null does not turn an equality query into a missing-field query.

Enterprise exception: Pipeline operations support EXISTS and IS_ABSENT. They can distinguish absent values on the server, including for updates targeting missing fields. Those are a different API from the Core queries above and require an Enterprise database. If your project uses them, consider that route before a client-side full scan.

Way 2: audit a field with a read-only Node scan

Scan by document ID, not the field you are investigating. Otherwise the scan itself excludes the documents you are looking for. This script reports missing values, explicit nulls and observed SDK value types. It keeps up to ten example paths per group, so its memory use does not grow with the whole collection.

Use Node.js 22 or later; this example was checked with firebase-admin 14.5.0. Create a fresh folder and install the Admin SDK:

npm init -y
npm install firebase-admin
export GOOGLE_APPLICATION_CREDENTIALS="$HOME/keys/my-project-reader.json"

The environment-variable command is Bash. In PowerShell use $env:GOOGLE_APPLICATION_CREDENTIALS = "C:\keys\my-project-reader.json". Replace the key path and project ID. A read-only IAM role such as roles/datastore.viewer is sufficient; Admin SDK reads bypass Firestore Security Rules.

Save as audit-firestore-field.mjs. The optional final argument limits the scan to a sample; omit it to page through the collection.

import { initializeApp, applicationDefault } from "firebase-admin/app";
import { getFirestore, FieldPath, Timestamp, GeoPoint, DocumentReference } from "firebase-admin/firestore";

const [projectId, collectionPath, fieldPath, sampleArg, ...extra] = process.argv.slice(2);
const max = sampleArg === undefined ? Infinity : Number(sampleArg);
if (!projectId || !collectionPath || !fieldPath || extra.length ||
    !/^[A-Za-z_][A-Za-z0-9_]*(\.[A-Za-z_][A-Za-z0-9_]*)*$/.test(fieldPath) ||
    (sampleArg !== undefined && (!Number.isSafeInteger(max) || max < 1))) {
  console.error("Usage: node audit-firestore-field.mjs <projectId> <collectionPath> <field.path> [sampleSize]");
  process.exit(1);
}
initializeApp({ projectId, credential: applicationDefault() });
const db = getFirestore(); // (default) database
const field = new FieldPath(...fieldPath.split("."));
const query = db.collection(collectionPath).orderBy(FieldPath.documentId());
const groups = new Map();
let cursor;
let scanned = 0;
let reachedEnd = false;

function typeOf(value) {
  if (value === undefined) return "missing";
  if (value === null) return "null";
  if (value instanceof Timestamp) return "timestamp";
  if (value instanceof GeoPoint) return "geopoint";
  if (value instanceof DocumentReference) return "reference";
  if (Buffer.isBuffer(value) || value instanceof Uint8Array) return "bytes";
  if (Array.isArray(value)) return "array";
  if (typeof value === "number") return Number.isFinite(value) ? "number" : "non-finite-number";
  if (typeof value === "object") {
    return value.constructor === Object ? "map" : "other-object";
  }
  return typeof value;
}

try {
  while (scanned < max) {
    const size = Math.min(200, max - scanned);
    const page = await (cursor ? query.startAfter(cursor) : query).limit(size).get();
    if (page.empty) { reachedEnd = true; break; }
    for (const doc of page.docs) {
      const type = typeOf(doc.get(field));
      const group = groups.get(type) ?? { count: 0, examples: [] };
      group.count += 1;
      if (group.examples.length < 10) group.examples.push(doc.ref.path);
      groups.set(type, group);
      scanned += 1;
    }
    if (page.size < size) { reachedEnd = true; break; }
    cursor = page.docs.at(-1);
  }
} finally {
  await db.terminate();
}
console.log(JSON.stringify({
  collection: collectionPath,
  field: fieldPath,
  scanned,
  scope: reachedEnd ? "reached-end" : "sample-limit",
  groups: Object.fromEntries(groups),
}, null, 2));

Run a small scan first, then the full collection if needed:

node audit-firestore-field.mjs PROJECT_ID products price 200
node audit-firestore-field.mjs PROJECT_ID products price
node audit-firestore-field.mjs PROJECT_ID users profile.email

For the three email documents above, the groups each contain one entry: string, null and missing. For a product migration, a number group alongside string is a useful signal to inspect those example paths. The script outputs paths and counts, not the field values themselves.

What the report proves

reached-end means the scan reached the end of that collection during this run. sample-limit means it stopped at your cap; it may not have checked the remaining documents. A sample ordered by ID is not random. At an exact sample boundary, the script conservatively reports sample-limit without making another request to prove that no documents remain.

Pages are separate reads, not one frozen snapshot. Concurrent inserts, edits or deletes can change the result while the script runs. It does not scan other collections or descendant subcollections. It uses the (default) database and accepts ordinary dotted paths such as profile.email; literal field names containing dots or other special characters need a deliberately constructed FieldPath instead.

The report classifies decoded Admin SDK values. Both integer and double fields appear as JavaScript numbers, nested map and array contents are not audited recursively, and vector values land in other-object. It is a field audit, not a complete Firestore wire-type schema.

Fix the data without inventing values

Treat each group separately:

• Missing: decide whether the field is optional or needs a trustworthy backfill. Do not use migration time as createdAt if the field is meant to describe the original creation time.
• Null: confirm whether this means unknown, deliberately cleared or not applicable. A blanket default can destroy that distinction.
• Mixed string and number: inspect the source format. Empty strings, decimal commas and units such as "12 EUR" need explicit handling; Number("") yields zero.

For a field rename, deploy readers that accept both names before removing the old one. For a correction, test representative records, rerun the audit, and keep a backup that can restore the original values. The bulk update guide covers field updates and a rename transaction that skips existing targets.

If missing-field queries are a recurring application requirement in Core, write an explicit queryable state such as emailState: "missing" or hasEmail: false alongside the data. Backfill older records and maintain that state in every writer; inconsistent markers create another schema problem.

Way 3: with Chiselbase

The script answers a precise question about one field. Chiselbase’s free schema map gives you a wider view while you investigate:

1. Open the collection or run a query. Observed documents feed the schema map using data already loaded; collecting those observations adds no extra reads.
2. Open Schema map and choose the collection. Inspect field presence percentages, type labels and common values. Look for a field that appears less often than expected, or use the mixed-types filter to find inconsistent types.
3. Use Rebuild for a fresh sample when the observations are old or too narrow. Review the collection scope and sampling settings; rebuilding fetches documents and costs reads.
4. Open a document or create a query from the schema view to investigate. Use the scan above when you need exact example paths for a missing field; the presence percentage itself is not a missing-document query.
5. Open TS or Zod and copy the inferred model as a starting point for validation in your application.

· schema map · orders

The map counts observations, including repeated reads of the same document. A presence percentage describes those observations, not necessarily the share of unique documents in the whole collection. A rebuild can replace accumulated observations with a fresh sample, but it still does not prove completeness across the database.

Generated types are inferred, not an enforced schema. Check the generated comments for fields omitted by the presence threshold. Review optional and nullable fields, model nested structures yourself, and adapt reference or bytes representations to your runtime. For example, an Admin SDK reference is a DocumentReference, while generated models may represent it as a string. Timestamp and GeoPoint types may need imports.

The practical benefit is staying in one workspace: a suspicious type in the map can lead straight to a document, a query and a reviewed correction. Schema inspection and code generation are Free features; bulk corrections use Pro.

Which way to pick

Keep the problem from returning

An interface alone does not validate stored data. Validate inputs in every writer, including Admin jobs and imports. For client writes, Security Rules can require fields with keys().hasAll(...) and check values with operators such as is number; see Firebase’s field validation guidance. Rules do not validate Admin SDK writes, so server-side validation still matters. Adding validation does not repair existing documents.

Read charges apply to every scan, including preview samples and repeat checks; applicable index-entry charges can also apply. Empty queries have a minimum read charge. For a large collection, choose the sample size deliberately and review Firestore pricing before repeated full scans.

Frequently asked questions

Can Firestore query documents where a field does not exist?

Core queries do not have an absent-field filter. Scan an appropriate collection without filtering on that field, or maintain an explicit state field for future queries. Enterprise Pipeline operations offer existence and absence functions.

Why does orderBy return fewer documents than the collection contains?

Documents missing the ordered field are excluded. A null value still counts as a present field. Compare a scan ordered by document ID with the field-based query to locate the difference.

Does a field present in every sampled document have to be required?

No. The sample may miss older records or another writer’s format. Decide what your application requires, then verify the data and enforce that contract. Chiselbase’s inferred model is evidence to review, not proof that every document complies.

Sources and example checks

Reviewed on October 4, 2026 against Firebase’s Core query behavior, ordering and existence, Pipeline existence functions, supported types and field validation. Chiselbase’s observation counts and generated-model behavior were checked against its schema implementation.

The published scan was exercised against the local Firestore emulator with missing, null, string, number, nested and special typed values, sampling and multiple pages. The query examples were checked for null and absent-field behavior. These checks do not verify production IAM, billing or Enterprise Pipeline execution.