Reported operating use · Aerospace data engineering

Eclipse Aerospace's reported flight-data ingestion workflow

A jointly engineered data connector brings existing decoders into Spark. This is reported data engineering; its proposed diagnostic rehearsal is separate.

Read proprietary flight files through a custom Spark connector

Eclipse Aerospace, as reported by Databricks

Vendor-reported customer-specific data engineering, published 12 November 2026. This is data automation, not an established AI deployment or flight-safety system.

A published third-party example, not a Nuvepro customer case or evidence of a partnership.

The workflow at a glance

Eclipse reports reading proprietary flight files inside Spark

Illustrative summary of the published account

  1. Flight-data source

    Land a telemetry file in cloud storage

    The runtime account starts with an original proprietary telemetry file in cloud storage.

  2. Next
    Registered PySpark source and Spark

    Schedule a lightweight reader on workers

    The custom source carries serializable input paths and read options; Spark schedules partitions rather than serializing the native decoder object.

  3. Next
    Worker and existing Rust decoder

    Decode bytes and return Arrow batches

    Each worker constructs the decoder inside read(). Rust returns Arrow RecordBatches through the C Data Interface.

  4. Next
    Spark and downstream data pipelines

    Consume batches into downstream tables

    Spark consumes Arrow batches into downstream tables. The account also reports visible decode errors and metrics, badRecordsPath capture and data-quality pipelines; reviewer and release authority are unspecified.

The sequence summarizes the file-to-table path explicitly described in the November 12 vendor account. Error visibility is not a post-write human approval. The Substack roundup displays November 14 despite its November 15 issue title. This is customer-specific data engineering, not an AI deployment. Code, performance and error handling were not independently run or audited. FDE-like is an inference from joint engineering, not proof of an FDE role, aviation-safety validation or customer acceptance. See the source account and limitations below.

What the source reports

Databricks reports joint work with Eclipse Aerospace to wrap its existing Rust decoders in a Python/Arrow connector. Spark schedules file inputs on workers, constructs the decoder there and consumes Arrow batches into downstream tables. Decode metrics and malformed records remain visible in the data workflow instead of a separate Lambda/Avro path.

Reported technology: Eclipse Aerospace and Databricks; Rust, PySpark custom data sources and Apache Arrow. Scope and outcomes are source-reported.

What the enterprise environment would need

The following is a proposed project design for an enterprise doing similar work. Published details are marked; the remaining systems, data and scenarios would be agreed for your project.

Proposed practice environment

Review a malformed flight-record diagnostic rehearsal

Illustrative project design to scope with your enterprise.

  1. 1 · Set up the environment

    Frozen synthetic bytes and independently decoded references

    Use permitted or synthetic telemetry fixtures, versioned format definitions, decoder and schema versions, and separately authored expected fields, units and error outcomes. Isolate the workspace from aircraft, production ingestion and maintenance systems.

  2. 2 · Preparation and agent contributions

    Agent prepares a source-linked diagnostic pack

    Draft a malformed-record explanation and simulated reprocessing plan, preserving file identity, source byte locations, decoder version and unresolved ambiguities. Produce reviewable diagnostics only; never rewrite telemetry or approve a real repair.

  3. 3 · Independent checks

    Independent engineer compares bytes and golden fields

    A verifier separate from the generating agent compares decoded field values, units and byte locations with independently prepared reference records. Check truncated, malformed, unsupported-version, duplicate and mismatched-schema fixtures plus the expected valid cases; preserve errors instead of fabricating corrected values.

  4. 4 · People decide acceptance

    Data-engineering owner accepts the exact diagnostic and simulated plan

    The accountable data-engineering owner accepts or rejects the exact independently checked diagnostic, simulated reprocessing plan, source fixtures, version references and unresolved errors. Actual reprocessing and connector repair remain separate engineering decisions; acceptance grants no aircraft or maintenance authority.

Only within the accepted practice scope
Practice output

A reviewable evidence package

A versioned diagnostic and simulated reprocessing plan with source-byte references, golden-record comparisons, unresolved errors and owner disposition.

If a check fails → withhold acceptance.

Changed format or decoder versions, missing references, conflicting fields, a failed golden-record check, an unauthorized data-write request or owner rejection withholds acceptance. Repair the diagnostic or fixture assumptions, repeat independent byte/field and error-case checks, then return the exact revised pack to the owner.

Failure or owner rejection returns to review

  1. Hold
  2. → 2 · Revise work
  3. → 3 · Rerun independent checks
  4. → 4 · Owner review

Operating boundary. Designed checks, not executed. Synthetic or approved rehearsal only; no telemetry correction, production reprocessing, flight-control action, maintenance sign-off or aviation-safety validation. Aerospace data specialists must confirm format references and acceptance rules.

Proposed, not an executed Nuvepro project or a description of the named company’s implementation. Owners agree the data, rules, responsibilities and acceptance criteria for their enterprise.
Synthetic telemetry and an independent decode reference
Proposed: provide permitted synthetic binary files, an agreed format specification, frozen decoder/environment versions and independently prepared expected field values. Keep original bytes immutable and all aircraft and maintenance systems disconnected.
Agent prepares a malformed-record diagnostic brief
Proposed: a read-only agent gathers failed-record offsets, parser messages, schema versions and fixture identities into a cited diagnostic draft. It distinguishes unsupported guesses from observed test evidence and never rewrites telemetry.
Independent field and failure-path checks
Proposed: a specialist separate from the drafting agent compares output fields, units, timestamps, record counts and rejected-byte locations with the format reference and independently prepared answers. Separately test that malformed records remain inspectable and failures are not silently dropped.
Engineering owner accepts a bounded reprocessing plan
Proposed: the accountable data-engineering owner accepts only the exact checked diagnostic and simulated reprocessing plan. A decoder, fixture or schema change invalidates earlier checks and returns through repair, independent replay and owner review.

A proposed simulation

An illustrative practice design derived from this work, not an executed assessment.

Rehearse a diagnostic and reprocessing handoff for synthetic telemetry files against an independent format and field reference.

Review gate: The engineering owner accepts only the exact diagnostic and simulated reprocessing plan whose fields, counts, errors and fixture versions passed independent comparison. This establishes no flight or maintenance authority.

Challenge the result: A truncated record is silently omitted or a decoder update changes a field unit. Hold acceptance, repair the test pipeline, repeat independent field/count/error checks and return the exact revision to the engineering owner.

What needs checking

  • Source boundary: joint customer-specific engineering supports an explicitly inferred FDE-like view, not a claim that the team held FDE titles. The FDE view links this same implementation rather than adding another deployment.
  • Proposed: test valid and truncated records, malformed lengths, duplicate files, schema drift, unit/timestamp errors and a decoder-package mismatch against a separately prepared format reference. Designed, not executed.
  • Proposed: change the decoder after an accepted diagnostic. Withhold acceptance, replay the fixed synthetic fixtures through independent checks and return the exact revised plan to the engineering owner. Designed, not executed.

Review environment fidelity, human capability and agent performance separately. The enterprise’s accountable owners decide acceptance of the shared workflow and any operating use.

How an enterprise could build similar work

A proposed build order for an enterprise doing similar work, written by Nuvepro from the published account and our interpretation of the environment and checks above. It is not how Eclipse Aerospace, as reported by Databricks built or operates this work.

  1. Synthetic telemetry and an independent decode referenceProposed: provide permitted synthetic binary files, an agreed format specification, frozen decoder/environment versions and independently prepared expected field values. Keep original bytes immutable and all aircraft and maintenance systems disconnected.
  2. Agent prepares a malformed-record diagnostic briefProposed: a read-only agent gathers failed-record offsets, parser messages, schema versions and fixture identities into a cited diagnostic draft. It distinguishes unsupported guesses from observed test evidence and never rewrites telemetry.
  3. Independent field and failure-path checksProposed: a specialist separate from the drafting agent compares output fields, units, timestamps, record counts and rejected-byte locations with the format reference and independently prepared answers. Separately test that malformed records remain inspectable and failures are not silently dropped.
  4. Engineering owner accepts a bounded reprocessing planProposed: the accountable data-engineering owner accepts only the exact checked diagnostic and simulated reprocessing plan. A decoder, fixture or schema change invalidates earlier checks and returns through repair, independent replay and owner review.

Your systems, data, rules and accountable owners decide the real project. Agree acceptance criteria, escalation owners and operating responsibility with the people accountable for the work before anything runs on live records. Treat the first run as a rehearsal on test records: compare the draft output with an independently prepared answer, and record what the people reviewing it change.

Questions about this example

Why is there also an FDE-like view?
Joint customer-specific engineering supports that interpretation. An explicit FDE role is not established, and both views refer to one implementation.
Is the proposed environment operating proof?
No. It is a separate fictional design. Its acceptance checks are designed, not executed, and do not establish production authority or customer impact.
Who accepts the proposed rehearsal?
In the proposed rehearsal, the engineering owner accepts only the exact diagnostic and simulated reprocessing plan whose fields, counts, errors and fixture versions passed independent comparison. This establishes no flight or maintenance authority.

Shape this around your enterprise

Bring a similar workflow and the people responsible for it. Together, define the records and systems to use, the result your team will build or configure, and how its owners will check it. Nuvepro scopes the simulation, environment and preparation. Agree any existing-agent evaluation, implementation assistance and production responsibilities separately.

Sources and limitations

Curation review: . This is the date this source example was reviewed, separate from source publication dates.

  • This vendor account is not an independent performance or cost audit. Code, benchmarks, decoded records and formal customer sign-off were not inspected or executed.
  • The linked ARINC 429 example does not establish Eclipse's exact proprietary format. Compatibility, quarantine/release authority and recovery responsibility need project-specific confirmation.
  • The 14 November Substack header differs from its 15 November issue title. The 12 November technical source is an explicit seven-day fallback, not proof of a newly launched implementation.
  • The proposed diagnostic environment is fictional and unexecuted. It grants no aircraft control, telemetry correction, maintenance approval or Nuvepro customer/partner status.
  1. Eclipse Aerospace simplifies flight-data ingestion with DatabricksSource publication: 2026-11-12
  2. The Weekly Industrial Stack · 15 November 2026Source publication: 2026-11-14

More software engineering examples

All 16 software engineering examples

Explore more enterprise work