29 Full-System Integration: The Goose Management System End to End
The previous chapter closed the book’s last open account and set the terms for this one: the welfare constraints ride along as requirements, checked where everything else is checked — in the code, in the parameters, and in the logs. What remains is the thing the Preface promised on its first page and every Part has advanced by one layer since: the whole system, running, once, end to end. The System Architecture chapter already did the anatomical version of this — seven nodes, ten services, eight topics, eleven states, every edge named and every trigger classified. This chapter does the physiological one. It follows a single flight of the Goose Management System from the operator’s service call to the disarm that closes the log, and at each phase asks not what fires the next transition — that trace is drawn, and will only be cited — but whose engineering is holding the aircraft up right now: which Part of this book built the machinery currently load-bearing, what it took to get right, and what would be going wrong at this exact moment if it hadn’t been.
A closing chapter owes more than a victory lap, so two harder questions frame the flight. The first is about testing: what does it mean to verify an assembly like this one? Every node in the graph has been exercised in its own chapter, most of them to destruction — and none of that, it turns out, establishes that the system works, because there is a whole class of failure that lives not in modules but in the seams between modules that are each, separately, correct. The engineering name for hunting that class is monolithic integration and stress testing, and the middle of this chapter gives it a concrete treatment using the book’s own machinery as both the examples and the defenses. The second question is about construction: how does a system like this come to exist at all? Not all at once. The running example was built the way working field systems generally are — as five incremental stages, each one a working, demonstrable system in its own right, each adding one layer to a foundation already proven. The order of those stages is a piece of transferable engineering doctrine, worth stating explicitly, because it will outlive this particular aircraft: the reader’s next autonomous system will not detect geese, but it will still need its telemetry proven before its autonomy and its failsafes proven before both.
29.1 Learning Objectives
- Trace a complete Goose Management System mission — takeoff, search, detection, harassment, an injected evasion event, and safe return — naming for each phase the real states and triggers established in the System Architecture chapter and the Part of the book where that phase’s internals live.
- Explain what monolithic integration and stress testing mean in practice for a multi-node ROS 2 system — merging the vision, safety, and control modules into one coherently running whole — and characterize the three failure species that only appear at integration time: convention mismatches, ordering and handoff failures, and shared-resource conflicts.
- Identify the running example’s designed defenses against each integration-time failure species — the centralized constants and frame modules, the ENGAGE_PREP warmup state, the active-controller token, and the single-writer tick — and explain why unit-level testing cannot substitute for any of them.
- Apply the five-stage incremental-delivery pattern — telemetry and environment, SITL and failsafes, live inference, simulated mission logic, field validation — as a general systems-engineering practice, and justify each of its ordering principles: infrastructure before autonomy, safety before capability, perception before decision-making, simulated logic before committed flight.
- Distinguish what simulation can validate from what only field validation reveals — GPS multipath near structures, lighting the detector’s dataset never sampled, radio-link degradation at range, real wind against the coverage budget — and pair each field-only failure with the machinery that predicts it and the log signature that diagnoses it.
29.2 Key Vocabulary
- Systems integration — the engineering phase in which separately developed, separately validated subsystems are combined into one running whole, and the interactions between them — rather than the subsystems themselves — become the object under test.
- Monolithic integration — the point at which every module runs together as a single coherent system: vision, safety, and control merged, all rates live, all resources shared. The word describes the test scope, not the software structure — the running example remains seven independent processes; what becomes monolithic is the thing being verified.
- Integration-time failure — a failure that exists only in the interaction between modules that each pass their own tests: a units or frame convention two nodes read differently, an ordering assumption each side believes the other satisfies, a race at a state boundary, a resource two correct processes both consume.
- Stress testing — deliberately operating the integrated system at and beyond its worst-case point — every module live, every rate at maximum, resources contended, failures injected — to surface behavior that appears only under load, where the margins that unit tests measured individually must all hold simultaneously.
- Incremental delivery — building a system as a sequence of stages, each independently a working, demonstrable system, so that integration risk is spent a little at a time and every new failure arrives with a one-layer suspect list rather than a whole-system one.
- Dry run — a complete end-to-end mission executed with the full system live, in the deployment environment where possible, before the flights that count; the unit of field validation.
- Field validation — the final validation stage, conducted in the real deployment environment, aimed specifically at the conditions simulation cannot faithfully produce: real GNSS propagation, real lighting, real radio behavior, real wind, and real animals.
29.3 One Flight, the Whole Book
Here is the flight. It is hypothetical only in its particulars; every state, trigger, threshold, and parameter in it is the real system’s, as the preceding chapters established them, and Figure 29.1 draws the whole arc with the provenance lane this chapter exists to add. Two configuration notes before the props turn, because both are the direct product of earlier chapters’ arguments. First, the harassment parameters are set through the ROS 2 parameter system into the design band — a 12 m orbit radius at 6 m altitude, speed under the 5 m/s group-proximity recommendation — and the behavioral termination rule runs alongside the shipped timer, exactly the configuration the Ethics and Ecology chapter’s review demanded and its Try It had the reader build. Their presence in the startup logs is the first checkable evidence that the previous chapter’s requirements genuinely ride along. Second, the flight carries the book’s own evasion layer — the Vector-Based Threat Evasion chapter’s evasion_command under the Finite State Machines chapter’s preemption discipline — attached at the seam the System Architecture chapter named for it. That honesty note gets its full paragraph when the threat appears.
Before the props turn, the flight is already running on Part I. The aircraft on the pad is that Part’s work standing still: the airframe the Anatomy of a Multirotor chapter sized against this mission’s payload, the battery whose limits the Materials and Power chapter established and Part I’s closing chapter converted into a mission time budget with an explicit reserve, the GNSS-IMU-barometer suite whose fusion the Sensing the World chapter followed from silicon flexure to attitude estimate. The operator’s pre-flight brief is the Airspace Law chapter compressed into a checklist — Part 107 currency, the visual-line-of-sight plan, the organization’s own ceilings sitting deliberately under the FAA’s — and the operator’s terminal session is the From Radio to Root Shell chapter’s whole thesis in one command: a secure shell into a Linux host that happens to be able to fly, with a MAVLink heartbeat as the first diagnostic, exactly as it was on the bench.
The ladder up is one operator call and three verified services. The operator calls /goose/start_mission, and the Mission Manager climbs the startup ladder the System Architecture chapter traced: PREFLIGHT passes only on a real GPS fix with at least eight satellites, battery at or above the 50 % launch floor — a threshold sitting far above PX4’s own failsafe tiers, the Geofencing and Fail-Safes chapter’s layered-margin doctrine made numeric — a valid local position, and a recorded home position; ARMING returns success only when PX4’s own telemetry shows the vehicle armed; START_MISSION confirms AUTO.MISSION the same way. Command verification, the Flight Controller’s one discipline, is why each rung of this ladder means something. And the moment the vehicle arms, a file begins: the autopilot’s logger opens the flight’s .ulg, the black-box recorder the Post-Flight Diagnostics chapter will read this evening whether or not anything goes wrong.
The search is Part IV’s arithmetic flown by Part II’s machinery. In MISSION, PX4’s onboard executor flies the pre-uploaded survey — the .plan file the Mission Scripting chapter learned to read field by field, its rows spaced by the Path Planning on the Edge chapter’s generator: a 90° camera at 5.5 m altitude with 40 % overlap demands legs at most 6.6 m apart, and the constant altitude is not a preference but the thing holding the swath width, and therefore the coverage guarantee, still. The companion computer streams nothing during this phase — the architectural choice the System Architecture chapter defended as robustness exactly where the mission spends most of its airborne time — but it is anything but idle. Part III’s pipeline runs continuously: capture, preprocess, infer, postprocess, publish, the latency budget the Real-Time Inference chapter itemized, with the detector holding its deliberately low 0.6 confidence floor because a missed goose costs coverage — while the Ethics and Ecology chapter’s caveat stands upstream of any commitment: the bar for initiating harassment sits far above the bar for merely investigating, because a false alarm at the engagement stage is not a wasted look but a delivered orbit. Every detection that survives the floor crosses the Coordinate Frames chapter’s mathematics — pixel and depth to camera point, to body, to local NED — before it becomes one hit in the Detector’s rolling buffer. And underneath everything, the Failsafe node ticks at 1 Hz, watching thresholds it will hopefully never cross.
The find takes about two seconds, on purpose. A flock enters the frame near the third leg’s far end. One glimpse makes the target valid; five hits inside two seconds, clustered within 3 m of their own median, make it confirmed — the two-boolean hysteresis that demands a track, not a bounding box, before the aircraft commits. The Mission Manager’s next tick reads the flag, checks that the engagement cooldown has elapsed, flips the token to 'navigation', and enters ENGAGE_PREP — where it deliberately waits. Offboard mode’s first hard rule, standing since the PX4 Autopilot Stack chapter, is that the setpoint stream must exist before the mode switch; Navigation, newly named by the token, is already streaming, and after the 1.0 s warmup mode_offboard succeeds on PX4’s own confirmation. From first glimpse to verified Offboard: roughly two seconds, almost all of it confirmation and warmup — delays the system chose, not delays it suffered.
The engagement is the book’s reactive machinery, each piece where it belongs. ENGAGE streams the approach at 20 Hz — the stream-a-setpoint, test-arrival mechanics built in Part II and relocated here, pointed at a standoff point recomputed from the flock’s tracked position — while the dead-man’s switch quietly becomes the phase’s safety net: silence for half a second would now be a mode change, which is why this phase is short and the search was not flown this way. Arrival flips the token to 'harassment', and HARASS flies the Harassment Pattern Generation chapter’s re-centering orbit around a flock center the Target Tracking and Visual Servoing chapter’s two-loop machinery keeps continuously fresh. The parameters flying are the field configuration, not the shipped defaults — 12 m and 6 m, the lateral geometry of the design band — and the engagement ends the way the Ethics and Ecology chapter argued it must: on the flock’s observed departure toward the open corridor, with the timer and the 90 s and 5 s watchdogs as backstops rather than as the rule. The geese lift, cohere, and move off along the corridor. The behavioral termination fires, harass_done publishes, and the engagement log records what the system believed it engaged, for how long, and with what response — the welfare record and the effectiveness record in the same rows.
Then the injected event: something turns toward the aircraft. Call it what the exercise scripts it as — a large bird peeling off the departing flock and closing. Perception delivers its track; the differenced world-frame positions give a relative velocity; and the three-part conjunction the Vector-Based Threat Evasion chapter derived — closing (\(\mathbf{r} \cdot \mathbf{v}_{rel} < 0\)), soon (\(t^*\) inside the horizon), and inside the safety radius — goes true at the estimator’s rate, not the detector’s. The preemption discipline does the rest: evaluated before the state dispatch on every tick, the trigger stashes the interrupted state and sweep index, slews onto the perpendicular escape vector, and climbs toward the safe altitude. The encounter clears — opening and beyond the radius with margin, hysteresis doing its quiet work — and the resume guard makes the honest choice the Finite State Machines chapter taught: the stashed harassment target is now a stale track, so control returns to the search, from which any fresh confirmation can re-enter the engagement through the front door.
One paragraph of candor, because this trace’s credibility depends on it. The shipped mission_manager.py contains no evasion state; the System Architecture chapter said so plainly, and nothing here retracts it. In the shipped machine, an uncooperative mover would have to reach the aircraft through the severity protocol — a watchdog publishing SEVERITY:reason, _enter_failsafe parsing it, the token zeroed, a recovery service called — which is a blunt but sound response: break off, don’t compute. The evasion phase this flight just flew is the book’s Part IV extension, installed at exactly the attachment point the System Architecture chapter identified — a new severity in the protocol, or a new state between the engagement watchdogs and RESUME. That such a layer can be added without touching Navigation, Harass, or the Flight Controller is not luck; it is the repository’s own claimed design benefit — a scalable architecture for adding future behaviors — and the claim is only testable at integration time, which is precisely why this chapter is where the extension finally flies.
The wind-down is deliberately boring. RESUME zeroes the token, re-enters AUTO.MISSION on a verified service call, and starts the 20 s cooldown that keeps the resumed sweep from instantly re-engaging the flock it just moved; had that service failed three times, the escalation path — RTL:cannot resume mission — stood ready. The sweep completes its remaining legs; the Mission Manager reads the end-of-search signature, in_mission seen and then in_loiter, and enters DONE. Return-to-Launch flies home on margins the Geofencing and Fail-Safes chapter sized from mission geometry, against the reserve Part I wrote into the time budget. Land. Disarm. The ULog closes, and the evening’s real work begins: the two recorders — the autopilot’s .ulg and the companion’s bag of /goose/* topics — correlated on one timeline, each authoritative for its own questions, exactly as the Post-Flight Diagnostics chapter prescribed.
Count what just flew. No phase of that flight was held up by one Part alone: the search was Part IV’s arithmetic in Part II’s executor watched by Part III’s pipeline through Part IV’s frames under Part V’s architecture, with Part I’s physics carrying all of it and the Ethics chapter’s constraints configured into the parameters. That entanglement is not a complication of the system — it is the system, and it is why the next section exists. Nothing about seven separately correct processes guarantees the sentence this paragraph just wrote.
29.4 Merging the Modules: What Only the Whole Can Fail
The Goose Management System’s verification story, module by module, is as good as this book could make it: control loops stepped and plotted, detectors validated against honest splits, planners checked against coverage arithmetic, state machines tortured at their boundaries, failsafes fired on purpose in simulation. Monolithic integration is the step all of that cannot substitute for: merging the vision module, the safety module, and the control modules into one coherently running system, everything live at once, and testing that — the whole as the unit under test. The word “monolithic” earns its place with a precision worth keeping: the software stays modular — seven processes, each separately crashable, which the System Architecture chapter argued is a feature — but the verification target becomes indivisible. A unit test validates a module against its own assumptions. Integration testing validates the assumptions against each other. The failures it exists to catch come in three recurring species, and the running example supplies a vivid instance of each — along with, in each case, a defense the book has already built, whose real purpose only becomes visible now.
Species one: convention mismatches. Two modules agree perfectly about an interface and silently disagree about what the numbers in it mean. The book’s sharpest standing example is the one the Coordinate Frames chapter drilled: local NED’s \(z\) axis points down, so altitude is negative and “climb to six metres” is \(z = -6.0\). Now imagine a contributed patch in which some node computes a setpoint altitude as a positive “metres up” and writes it into the \(z\) field unnegated. That node’s unit tests pass — its arithmetic is self-consistent. The consumer’s tests pass — it faithfully flies whatever \(z\) it is handed. The integrated system commands a descent into the ground, and only a test that runs both modules against one simulated aircraft can see it. The same species covers degrees handed to a radians consumer, a message stamped in one clock read against another, and — the documentation variant the System Architecture chapter catalogued — a README asserting token values the code never publishes. The defenses are centralization and diff-testing: one frames.py as the single authority for every transformation, one px4_defs.py as the single authority for every topic name and constant, so a convention has exactly one place to be right; and the habit of arbitrating documentation against running introspection — ros2 topic list outranks the README — because prose conventions decay and code conventions bind.
Species two: ordering and handoff failures. Each module’s contract is satisfiable; the sequence of satisfying them is not, or is racy. The running example’s canonical case is the Offboard handoff, and it is worth savoring because the shipped system contains a whole state that exists only to solve it. PX4 refuses to enter Offboard mode unless a setpoint stream is already flowing — the dead-man’s rule from Part II. A behavior node, correctly, streams only while the token names it. Sequence those two correct rules naively — switch the mode, then grant the token — and the mode switch is refused every time; grant and switch in the same instant and it is refused some of the time, which is worse, because intermittent integration failures are the expensive kind. ENGAGE_PREP is the integrated answer: token first, a 1.0 s warmup during which Navigation’s stream establishes itself, then the verified mode switch. No unit test of the Flight Controller or of Navigation could ever demand that state’s existence; only the pairing does. The same species includes the race conditions the Finite State Machines chapter dissected — the threat trigger landing on the exact tick of a grid-corner arrival, callbacks writing state out from under a half-finished transition — and the defense is the same one taught there, now legible as an integration measure: the single-writer tick, all triggers evaluated in one fixed priority order, at most one transition per tick, so that simultaneity is resolved by decree rather than by scheduler accident.
Species three: shared-resource conflicts. Two correct modules consume one thing that cannot serve both. The aircraft itself is the system’s most important shared resource, and the clearest way to see the token’s real job is to delete it: Navigation and Harass are both, by their own specs, entitled to stream setpoints; run both without arbitration and PX4 receives two interleaved 20 Hz streams commanding different points — an aircraft lurching between two masters, each master’s unit tests green. The /goose/active_controller token is mutual exclusion by broadcast: one writer, one named node, checked at the top of every behavior tick, revocable by decree. The subtler shared resource is the companion computer’s processor. Part III’s inference, Navigation’s 20 Hz stream, the Detector’s frame conversions, the bag recorder, and the Mission Manager’s tick all buy their guarantees from the same CPU — and the dead-man’s switch prices CPU exhaustion in flight modes: stretch the setpoint period past PX4’s tolerance and a saturated processor becomes an Offboard drop mid-engagement. The Real-Time Inference chapter’s benchmarking discipline — measure on the target hardware, under the mission’s real competing load, long enough to catch thermal throttling — was this species’ unit-level defense; the integration-level one is the stress test.
That is what stress testing means for this system, and it is more specific than “run it hard.” Assemble the worst honest case: the full graph live, the detector inferring on real imagery at full rate, an engagement active so the 20 Hz stream is load-bearing, the bag recording everything, on the actual companion computer at its actual thermal steady state — and then measure the margins that unit tests measured separately, all at once: setpoint-period tail latency against the Offboard tolerance, detection latency against the reaction budget, tick jitter against the transition-latency bound. Then inject the failures the architecture claims to absorb, in the worst phase: kill the Detector mid-engagement and time the 5 s target-lost watchdog; freeze telemetry and confirm the staleness ladder holds and then goes home; script the threat onto the exact tick of a state boundary, the torture case the Finite State Machines chapter designed. Each of those was rehearsed somewhere in this book against one module. The integration version asks the only question the field will ask: do all the margins hold simultaneously, when every subsystem is spending from the same budgets? A system that passes fails the way the Geofencing chapter said good systems fail — boringly — and a system that fails has told you, on the bench, what would otherwise have been a field incident. Either answer is a bargain, and the price of admission is simply doing it: everything on, everything measured, everything injected, before the flight that counts.
29.5 Five Deliveries, Each One a Working System
How does a system that must eventually pass that test come to exist? The tempting answer — build all the modules, then merge them — has a name in the systems-engineering literature, big-bang integration, and a well-earned reputation: it defers every seam failure to the end, then delivers them all at once, into the largest possible search space, at the moment of maximum schedule pressure. The Goose Management System was built the other way, as five incremental stages, and Figure 29.2 draws the staircase. The pattern is worth studying not as this project’s history but as a reusable shape: each stage is the cheapest, most isolated piece that can be made to work next, each is independently a working system someone could demonstrate, and each becomes the proven foundation the next stage stands on.
Stage 1 — telemetry and environment. The companion computer running on the airframe, a remote shell into it, and real sensor data confirmed flowing and logged. Nothing flies; nothing decides. It is also the stage nothing above can be trusted without, because every later stage will be debugged through this layer: the topic echoes, the logs, the shell session. This is Part I’s closing chapters as a delivery — the From Radio to Root Shell chapter’s bench verification, heartbeat first, promoted to the aircraft — and it encodes the first ordering principle: infrastructure before autonomy. An autonomy bug on top of an unproven data path is two bugs that look like one.
Stage 2 — SITL and safety failsafes. Waypoint commands flying in software-in-the-loop, with the geofence armed and a kill-switch routine — the abort path — demonstrated, before any live inference or autonomy logic exists. This is Part II delivered whole: the three-process workbench, the mission as data, the fence with consequences, the graded failsafe ladder. The ordering principle here is the pattern’s least intuitive and most important: safety before capability. The fence and the abort are cheap to build now, when there is nothing dangerous for them to constrain, and nearly impossible to retrofit honestly later — a failsafe bolted onto an already-flying autonomous system inherits every architectural assumption that system made without it, and the Geofencing chapter’s default-safe doctrine cuts the same way: the constraint must be in place before the thing it constrains, or the thing defines the defaults. Note also what Stage 2 already is: a working survey aircraft. The Remote Sensing at Scale chapter flew serious agricultural missions on exactly this capability set — coverage flight plus a payload — with no goose, no detector, and no state machine anywhere in sight. A stage is a system, not a fragment.
Stage 3 — the live inference pipeline. Dynamic camera-frame ingestion through an optimized edge model, identifying targets on the real hardware — the first stage where perception stops being scripted. Part III delivered: the pipeline, the quantized and compiled model, the honest benchmarks on the target processor, and the frame chain from the Coordinate Frames chapter turning boxes into world points. The principle: perception before decision-making. The confirmation thresholds, persistence windows, and confidence floors that Stage 4’s state machine will depend on can only be sized against a real detector’s real false-positive and false-negative behavior; tune them against simulated perfect detections and every guard in the mission logic learns numbers the field will refuse to honor.
Stage 4 — simulated mission logic. The comprehensive state machine, in SITL: executing the search grid, transitioning to the harassment pattern, computing an evasion vector — deliberately still in simulation, with all of Part IV live at once: planner, frames, patterns, tracker, FSM, evasion, and the token discipline of the eventual architecture. The principle is the simulate-then-fly rule of Part II, applied now at mission scale: simulated logic before committed flight. This is where the integration failures of the previous section are cheapest to find — the torture cases, the boundary races, the handoff ordering — because a crash costs a restart, and because the diff from Stage 3 is exactly one layer: if Stage 4 misbehaves, the perception it consumes and the flight envelope it flies in are already proven, so the suspect is the new decision logic and nothing else. That one-layer suspect list is incremental delivery’s whole economic argument, and it is why the stages are ordered by dependency rather than by glamour.
Stage 5 — live field validation. The full autonomous loop — every prior stage’s work, integrated — flown at the actual deployment site. It is the first moment everything must work together outdoors, and the pattern’s last principle is a double negative worth untangling: real-world integration comes last, but integration risk is not deferred to it. By the time Stage 5 flies, every seam has already been exercised somewhere cheaper; what remains for the field is the set of conditions no simulator was asked to produce. That set is real, structured, and predictable enough to deserve its own section.
29.6 What Only the Field Can Teach
The field is not a harsher simulator. It is a different epistemic regime: the place where conditions nobody modeled — because nobody thought of them, or because modeling them faithfully is harder than encountering them — finally get their vote. Field validation is therefore not a rerun of Stage 4 with grass; it is a distinct activity with its own target list, conducted as complete end-to-end dry runs at the deployment site, tuning flight behavior against real conditions and finalizing the edge-case parameters that only reveal themselves outdoors. Four entries recur on that list for a system like this one, and the book has, without always saying so, already armed the reader against each.
Real GNSS propagation. Simulation’s GPS is a clean constellation; the deployment site has a tree line, a maintenance shed, and a pond — surfaces that reflect satellite signals into the receiver alongside the direct path. That multipath corrupts pseudoranges and sets the position estimate wandering by metres precisely where the site’s structures are, which is often precisely where the geofence margin is thinnest. The system’s layered defenses were built chapters ago — EKF2’s innovation monitoring down-weights a satellite stream that stops agreeing with the inertial evidence, and the Post-Flight Diagnostics chapter’s GPS-glitch signature (satellite-count drops, position jumps against a smooth IMU) makes the event legible in the log — but the parameter consequence is a field decision: the fence margins and arrival radii that were comfortable under open-sky scatter get re-sized against the site’s measured worst corner. That is what “finalizing edge-case parameters” means in practice: the number was always a placeholder for a measurement the bench could not make.
Real light. The detector’s validation score was earned on its dataset’s distribution, and the Real-Time Inference chapter was blunt about what happens when the field leaves it: the score silently stops predicting. Dusk shifts the color balance and stretches exposures into motion blur; low sun drapes half the pond in shadow; none of it announces itself, because the model keeps emitting confident boxes either way. Field validation is where the per-condition evaluation table gets its field-collected rows — morning, noon, overcast, dusk — and where the runtime health monitoring that chapter demanded proves it can notice a sagging confidence trend and degrade gracefully instead of steering on fiction. A dry run that includes a deliberately marginal-light flight is not pessimism; it is the only honest way to learn where the mission’s operating window actually closes.
Real radio. The bench link of Part I was a metre of wire; the field link crosses hundreds of metres, vegetation, and the site’s own reflections, and degrades with all three. The link-loss failsafe has been configured since Part II — but field validation is where it gets rehearsed at range, and where an operational fact gets checked that is easy to overlook on the bench: the operator’s two buttons, /goose/start_mission and /goose/abort, ride the network too. An abort path that works only where the link is strong is an abort path that fails exactly when the aircraft is far, low, or behind the shed — so the dry-run script includes calling it from the worst corner of the site, and timing it.
Real wind. Gazebo’s optional wind is a disturbance model; the site’s wind is gusts, gradients, and rotor from the tree line. Part II’s controllers were designed for disturbance rejection and Part IV’s coverage arithmetic budgeted the cross-track error that keeps adjacent swaths overlapping — but the budget’s numbers came from assumptions, and the field flight is where the assumption meets the anemometer. The tuning loop is the Post-Flight Diagnostics chapter’s workflow run at tempo: fly the sweep in the morning’s real wind, pull the cross-track series from the log, check it against the overlap margin, adjust, re-fly. If the margin holds in the site’s typical worst wind, the coverage guarantee survives contact with the world; if it does not, the honest fixes are the planner’s — tighten the spacing, accept the longer track — before they are the controller’s.
Two habits make these dry runs compound rather than merely accumulate. First, every anomaly becomes a log artifact and every log artifact becomes a hypothesis — the review-hypothesize-bench-test-fix-refly discipline, applied now to the integrated whole, with both recorders correlated so that a mission-level symptom can be traced to an autopilot-level cause. Second, the constraints that rode along get checked too: the field is where the Ethics and Ecology chapter’s behavioral termination meets actual geese, where the flock’s real responses populate the engagement log’s welfare fields, and where the parameter file that was audited on the bench is confirmed — from the logs, not from memory — to be the configuration that actually flew. The field validates the system; the logs validate the field work; and the same evening ritual that closes each dry run is the one that will close every operational flight after it.
29.7 Check Your Understanding
- Using only numbers established in this Part — detection hits landing at 5 Hz, five clustered hits to confirm, Detector publishing at 10 Hz, Mission Manager ticking at 10 Hz, a 1.0 s Offboard warmup, and (given for this problem) a confirmed target 18 m away approached at 3 m/s to a 3 m standoff with a 2.0 m arrival radius — estimate the time from a goose’s first detected frame to the token flipping to
'harassment'. Which terms dominate, and what does that say about where integration effort should go?
Confirmation: four inter-hit intervals at 5 Hz \(= 0.8\) s; up to one Detector publish period (\(0.1\) s) plus one manager tick (\(0.1\) s) before the flag is read — call it \(1.0\) s to enter ENGAGE_PREP. Warmup and verified mode_offboard: \(1.0\) s. The approach must close from 18 m to within 2.0 m of the 3 m standoff point, roughly 13 m of flight at 3 m/s \(\approx 4.3\) s. Total \(\approx 6.3\) s, of which the flight itself is about two-thirds and the two deliberate delays (confirmation, warmup) most of the rest; middleware latency is noise. The lesson generalizes: the integrated system’s end-to-end tempo is dominated by physics and by chosen safety delays, not by message plumbing — so integration effort belongs on verifying that the chosen delays are actually enforced (the warmup really precedes the mode switch; confirmation really precedes commitment), not on shaving ticks.
- Classify each incident as a convention mismatch, an ordering/handoff failure, or a shared-resource conflict, and name the running example’s designed defense: (a) a new deterrence node publishes trajectory setpoints whenever it sees a confirmed target, and the aircraft occasionally lurches between two flight paths; (b) a patch computes the approach altitude as \(+3.0\) “metres up” and writes it directly into the NED \(z\) field; (c) a “snappier” Mission Manager calls
mode_offboardon the same tick it grants the token, and the switch is refused roughly one time in five.
- Shared-resource conflict — two behavior nodes streaming to one autopilot. Defense: the active-controller token; the new node must gate every tick on being named, and the Mission Manager remains the token’s only writer. (b) Convention mismatch — a self-consistent module handing a positive-up altitude to a \(z\)-down consumer, commanding descent underground; both modules’ unit tests pass. Defense: centralized conventions — all altitude handling through
frames.pyand the shared constants module — plus an integrated SITL flight, the only test that runs both sides against one simulated aircraft. (c) Ordering failure — PX4’s stream-before-switch rule racing the token grant; the intermittency is the tell, because whether the first setpoints beat the mode command depends on scheduling. Defense: ENGAGE_PREP itself, the state that exists to sequence the handoff — token first, warmup, then the verified switch.
- A teammate argues the delivery order should start with Stage 3: “the detector is the highest-risk item, so de-risk it first.” Steelman the proposal, then defend the pattern’s actual order using at least two of its principles.
The steelman is real: schedule risk concentrates in the hardest component, and detector work (data collection, training, optimization) has long lead times — and indeed nothing stops model development from proceeding in parallel on recorded data. But delivering live inference first inverts two principles. Infrastructure before autonomy: Stage 3’s definition is inference on the real hardware with real frames — which requires the companion computer, the camera path, the logging, and the remote access that Stage 1 exists to prove; attempted first, every model bug is indistinguishable from a plumbing bug. Safety before capability: Stage 3’s output will immediately create pressure to fly against it, and flying anything before the fence, the failsafes, and the abort exist means the constraint layer gets retrofitted around a system already shaped without it — the exact bolt-on the Geofencing chapter’s doctrine forbids. The pattern’s answer to the schedule concern is not reordering but parallelism: train offline early; integrate in dependency order.
- Four anomalies appear during field validation that never appeared in SITL: (a) the position estimate wanders near the maintenance shed; (b) recall collapses on the evening dry run; (c) the abort call times out at the site’s far corner; (d) adjacent swaths lose overlap on a gusty afternoon. For each, name the field condition, the chapter machinery that anticipated it, and the log evidence that diagnoses it.
- GPS multipath off the structure: reflected signals corrupt pseudoranges. Anticipated by EKF2’s innovation monitoring (Part I) down-weighting inconsistent GPS; diagnosed by the Post-Flight Diagnostics chapter’s GPS-glitch signature — satellite-count drops and position jumps unsupported by IMU evidence. Consequence: re-size fence margins near the shed. (b) Distribution shift, lighting column: dusk lies outside the training distribution, so the validation score silently stops predicting (Real-Time Inference chapter); diagnosed by sagging confidence and detection-rate trends in the bagged Target stream, and answered by per-condition evaluation rows and graceful degradation, then targeted evening data collection. (c) Radio-link degradation at range — the operator’s service calls ride the same network as everything else; anticipated by the link-loss failsafe (Part II), diagnosed by comparing the call’s send time against the manager’s log, and fixed operationally: relocate the ground station or shrink the operating area. (d) Real wind against the coverage budget: the cross-track error series pulled from the ULog, checked against the overlap margin the Path Planning chapter budgeted; if it exceeds the margin in typical site wind, tighten the sweep spacing — the coverage guarantee, not aesthetics, is what is at stake.
- During HARASS, a scripted intruder begins closing on the aircraft. Contrast what the shipped system does with what the system flown in this chapter’s trace does, and name the exact attachment seam that separates them.
The shipped machine has no evasion state and no threat trigger; its reactive-safety path is the severity protocol. Unless a watchdog translates the intruder into a SEVERITY:reason message, the shipped system continues its orbit until harass_done or a watchdog fires — and if a severity is published, _enter_failsafe parses it, zeroes the token (silencing Harass within a tick), and calls the matching recovery service: a blunt break-off, terminal for the mission. The traced system carries the book’s Part IV extension: the three-part conjunction (closing, soon, inside the safety radius) evaluated at estimator rate in the tick’s preamble, preemption with a stash, a perpendicular escape climb, a hysteretic clearance test — and a guarded resume that returns to the search, not the harassment, because the stashed target track has gone stale. The seam is the one the System Architecture chapter named: a new severity in the SEVERITY:reason protocol, or a new state between the engagement watchdogs and RESUME. The contrast is the point: both responses are defensible engineering; only one of them is in the repository, and an integrated test is where the difference stops being theoretical.
- Design the single most informative stress test for the claim “the engagement loop holds under full load.” Specify what runs, what is measured against what bound, and which of the three integration-failure species it hunts.
Run the whole graph on the real companion computer at thermal steady state (a long warm-up first, per the benchmarking discipline): detector inferring on real recorded imagery at full rate, an active engagement so Navigation’s 20 Hz stream is load-bearing, the bag recorder capturing all topics, Mission Manager ticking at 10 Hz. Measure tails, not means: the setpoint inter-publish interval against PX4’s Offboard timeout (the dead-man’s floor of roughly 2 Hz), end-to-end detection latency against the reaction budget, and tick jitter against the transition-latency bound — each margin was verified separately in its own chapter, and this test asks whether they hold simultaneously. It hunts species three, shared-resource conflict: the CPU (and its thermal envelope) is the contended resource, and the failure it would expose — a stretched setpoint period dropping Offboard mid-engagement — is invisible to every per-module benchmark because each module, tested alone, has the whole processor to itself.
29.8 Try It
Part 1 — deliver the system, in the stages’ order. This is the book’s closing exercise: stand up the full Goose Management System — the shipped repository if you have it, or your own Part II–IV implementations against the System Architecture chapter’s interface table — and bring it to a complete end-to-end mission, delivering it to yourself in the five stages rather than launching everything at once. Stage 1: SITL (or the real companion computer) up, ros2 topic hz proving the telemetry streams, one arm-disarm cycle proving a ULog gets written. Stage 2: the lawnmower .plan flying in AUTO.MISSION with the geofence armed and one deliberate breach demonstrated, plus /goose/abort called mid-flight and timed — before any autonomy node runs. Stage 3: the Detector live against scripted or recorded imagery, /goose/target at 10 Hz, and a logged measurement of its confirm latency and false-alarm behavior. Stage 4: the full mission machine in SITL — and, at the seam the System Architecture chapter named, the evasion layer from Part IV. Then the full trace, as one flight: startup ladder, sweep, scripted goose, engagement, behavioral termination (implement it beside the timer if you built the Ethics chapter’s Try It — this is where it finally flies), an injected threat during HARASS, resume, sweep completion on the in_loiter signature, RTL. Before takeoff, audit the parameters as a field crew would: ros2 param get on the harassment node must show the design band, not the defaults, and the startup log must record it. The acceptance test is the trace itself: every transition in the Mission Manager’s log matched to its trigger and checked against the System Architecture chapter’s figure, with no transition you cannot explain.
Part 2 — stress it, then read it back. With the full mission running, apply the load and the knives: record everything with the bag recorder while the detector runs at full rate, and capture the setpoint-period and detection-latency tails during the engagement (on a desktop SITL host these margins will be comfortable — state them anyway, and note which would tighten on embedded hardware and by roughly how much, using the Real-Time Inference chapter’s measured ratios). Kill the Detector process mid-engagement and time the target-lost watchdog; freeze the telemetry bridge and watch the staleness response; re-inject the threat on the exact tick of a leg-corner arrival, the race the Finite State Machines chapter taught you to script from the arrival guard’s own log line. Then close the loop the way every field evening should: correlate the ULog and the bag on one timeline and write the flight’s one-page account — every state change with its trigger, every injected fault with the response and its measured latency, the engagement’s welfare-relevant fields (what was engaged, for how long, ending on which criterion), and the parameter configuration that actually flew, read from the logs rather than from memory. If that page is boring, the system passed. File it the way the Post-Flight Diagnostics chapter taught you to file everything: as the baseline the next flight’s anomalies will be diffed against.
Related Lab: See Appendix B for the lab assignment that applies this chapter’s concepts.
The Preface made one promise about the running example: that each part of the book would pick the thread back up, so the reader could see exactly where the material just read lands in a working system. This chapter is where the threads stop being picked up one at a time, because they are all load-bearing at once — which was the point of choosing a system, rather than a topic, as the book’s spine. The Goose Management System itself was never the destination. It is one modest aircraft with one eccentric mission, its swoop pattern still unbuilt, its evasion layer still the book’s extension rather than shipped code, its multi-aircraft future still the design exercise the Multi-Agent Systems and Coordination chapter closed Part IV with — the normal condition of a real system, which ends its chapters mid-story or not at all. What the system was, from the Commercial Drone Landscape chapter forward, was a forcing function: a single concrete artifact demanding that control theory, middleware, perception, planning, safety engineering, and ethics all be true at the same time, which is the one demand no sequence of isolated topics ever makes.
What the reader keeps is correspondingly not a drone but a competence the Preface called by name: the ability to reason about an autonomous aerial system end to end, and to know where to look when it misbehaves in the field. Size the airframe against the mission; prove the data path before the autonomy and the failsafe before the capability; buy perception’s numbers honestly, on the target hardware; keep decisions in one tick and conventions in one file; deliver in working layers; test the whole as a whole; read the logs every evening; and hold the system answerable — in its parameters and its records — to every party in the frame, including the ones with feathers. Those habits transfer to systems that will look nothing like this one, and that is the real closing state of the book: not a finished aircraft, but an engineer who knows, next time and from the start, in what order a working system is built.