To answer my own question, the trick is to leave the irrelevant fields empty, e.g. This document describes one campaign and one ad group.
Bid Strategy Type,Budget,Campaign state,Campaign subtype,Campaign type,Campaign,Ad group state,Ad group,Default max. CPC
cpc,1000000,enabled,Standard,Search Only,Poster Boys (2017),,,
,,,,,Poster Boys (2017),enabled,Poster Boys (2017),200000
Notice that the ad group describing row excludes campaign specific field values.
Refer to https://support.google.com/adwords/editor/answer/56368 for additional documentation on bulk uploads.
Somewhat off-topic, but it wasted me good 30 minutes.
When using this approach and using "Upload and preview" to test, you will get multiple errors such as "Item not found" and "Could not find the specified Ad Group". Ignore these errors and proceed to commit the changes. The errors are a red herring.