Skip to content

Consent-aware export — sharing recordings off the acquisition PC

An export hands a subset of the lab's dataset to someone else: this team's future work, or an open repository. Two things decide what goes. Which sessions is decided by each session's data-use stamp (its purpose and tier, the data_use sidecar block), re-read from disk at the moment of export (#17), and every pilot / development session is also held back, under either purpose (§1). Which files and fields is decided by one allow-list, the same for both purposes but for two omissions a public share makes (§3, §6) (#296). AGENTS.md §4 requires honouring consent on every export or share, mechanically rather than by someone remembering. Code: src/ssvep/io/export.py; UI: the Export… row on the Analyze tab of ssvep-toolbox.

This is a different operation from the auto-push to the lab server. Push mirrors a finished run, whole, to the lab's own server as internal storage. Export writes a new, reduced dataset out, for reuse or public sharing, into a folder the operator picks. The two share one discipline (re-read the data-use stamp from the sidecar on disk, never trust a caller-supplied list) but ask the matrix different questions: push asks whether a recording may be mirrored to the lab's own storage, export whether it may be handed on for this purpose.


1. The data-use stamp → purpose mapping

Two purposes, in a dropdown on the Analyze tab. Each is one action in the data-use matrix (ssvep.runtime.data_use.MATRIX, the policy every gate reads; see docs/DATA_GOVERNANCE.md, "Session purpose and data-use tier decide what may happen to a recording"), and the gate reads each session's data-use stamp: the data_use block every sidecar the session wrote carries, set when the session was locked (#286):

Purpose (dropdown label) export module constant Needs, in every sidecar of the session
Future use (internal reuse) export.FUTURE_USE a research stamp at tier 2 or 3
Public share export.PUBLIC_SHARE a research stamp at tier 3

A pilot, teaching or bench recording is never exported, under either purpose. An observer-only session (tier 0) wrote nothing, so there is nothing of it to export.

Missing or unreadable is always "no", never "yes":

Situation Exported?
every *.session.json and *_beh.json in the session carries a stamp the matrix allows for this purpose ✅ yes
a research stamp at a tier below the purpose's (tier 1 for either; tier 2 for a public share) ❌ no — not allowed for this purpose
a pilot, teaching or bench stamp (any sidecar in the session) ❌ the whole session is held back, under either purpose
no stamp: only the older three-flag consent record, or nothing ❌ held back — there is no migration; the three flags are never translated into a tier
a stamp that cannot be read: an unknown key or purpose, a tier of the wrong JSON type ("2", 2.0, true), a missing field, tier 0 ❌ held back — never a grant; the reason names the problem and the sidecar
a stamp and a legacy consent record in one sidecar ❌ held back — which one holds is ambiguous
no *.session.json sidecar, or an unreadable one ❌ held back — data use unknown
a subject code that isn't sub- plus digits, or a label that reads as a date ❌ held back
a BIDS-view session (sub-XXX/ses-YYY/) whose sourcedata/ session is gone ❌ held back and listed — data use unknown

Every sidecar in the session counts: each run's (a rescued *.rescued-NN.session.json included) and each questionnaire answer's. The most restrictive one decides. The stamp is read by the one fail-closed reader, ssvep.runtime.data_use.data_use_of, which push shares (ssvep.runtime.push.consent_block_reason).

The gate is per-session, not per-subject: a subject with two sessions, one consented for public share and one not, exports only the consented one. participants.tsv then lists only subjects with at least one exported session (§3).

Pilot / development sessions

A session declared a pilot / development session at lock (its recordings' data-use purpose pilot, chosen on Set up Session, or the older session_position.pilot, #161) is not study data. The matrix already refuses a pilot stamp; the export names the reason the same way for every pilot session, old or new, and decides about it before the data-use gate, at session granularity, by the one rule the converter's session_type column, the demographics fill and every contrast read too (ssvep.io.bids.not_study_sessions): a session is pilot when any of its recordings declares it (ssvep.io.bids.is_pilot_session; only the string "pilot" or a JSON true counts). export.pilot_session_keys reads that declaration two ways, and a session counts if either says so: the converter's reading (each recording's sidecar, keyed by the recording's file name) and every sidecar in the session's sourcedata/ folder, which is what the export gates by and which needs no XDF: each recording's *.session.json (a rescued one included) and each questionnaire answer's *_beh.json, which carries the same stamp when it was given in a session locked as pilot. (#371, #378)

A session declared pilot or test after it was recorded (its declarations file, python -m ssvep.io.declarations … --group pilot, see the BIDS guide's Participant declarations) is held back exactly like one locked as a pilot, under both purposes, and listed with export.DECLARED_WHY (#400). So is a session whose declarations file cannot be read (export.UNREADABLE_DECLARATIONS_WHY), since it may say pilot. Only the declared session is held back; the participant's other sessions are decided on their own.

Every pilot session is held back, under both purposes, and listed in ExportResult.excluded with export.PILOT_WHY. A participant whose every session is pilot loses their participants.tsv row, under either purpose, and the row is named in ExportResult.refused. No sub-XXX_sessions.tsv / .json is ever written (§6).

A participant with one pilot and one study session (mixed) keeps their row under both purposes: only the study session is exported. Nothing of a held-back session leaves by another route either: its questionnaire ratings stay out of phenotype/, and its pictures are not shipped. The converter's own bookkeeping, SSVEPConverterPilotGroups in dataset_description.json (which participants' pilot group it set itself, by code), is never exported under either purpose, and neither is its SSVEPPurposes list, which names the purposes of every recording on the lab's side, the held-back ones included. A dataset with no pilot session exports byte for byte what it did before this gate existed, with or without those keys and the converter's sessions table. A study's dataset (docs/SITE_PROFILE.md §10a) carries its study id, written by the export from the tree itself rather than copied: SSVEPStudy in dataset_description.json and the root's .ssvep-study.json marker, so the exported copy says which study it is. A dataset with no study carries neither.

A session with a mixed run (one run declared pilot, another not) is held back as a whole, under either purpose: the gate is per session, matching the converter's own session granularity (#360), even where the lab's own contrast keeps the non-pilot run.


2. What an export contains

export.export_dataset(bids_root, destination, purpose, *, include_stimuli=False, archive=False) writes, into destination:

  1. Per included session, its BIDS view, regenerated: sub-XXX/ses-YYY/eeg/ (BrainVision .vhdr/.vmrk/.eeg, _eeg.json with the run's stimulus design (§10), _channels.tsv, _events.tsv, _events.json, and _electrodes.tsv/_coordsystem.json where the montage has positions) and sub-XXX_ses-YYY_scans.tsv. An export never derives anything itself: a run with no BIDS view yet is listed in ExportResult.unconverted and left out (run the Analyze tab or code/01_xdf_to_bids.py first).
  2. phenotype/: the questionnaire ratings, one table per instrument with a JSON data dictionary (§9).
  3. Dataset-level files, regenerated: dataset_description.json, participants.tsv (filtered, §3), participants.json, the task-*_events.json of the tasks exported, channels.json, and a README the export writes itself. No sub-XXX/sub-XXX_sessions.tsv / .json is written: its session_type records each session's purpose on the lab's side, and an export carries research sessions only (§1, §6).
  4. stimuli/, only when asked for that export (§8).

Nothing from sourcedata/ is exported, under either purpose. Not the XDF, not the *.session.json (which carries the operator's identity and free-text notes), not the impedance log, not the questionnaire files as written, not a session's copies of its pictures. The BIDS view already carries the EEG, channels, electrodes and events, and the team already holds full sourcedata on lab storage. A redacted sourcedata copy would need a second allow-list kept in step with the first, for no content the dataset promises.

Two things are read from sourcedata/ and written out fresh, never copied: the questionnaire ratings (§9), and each run's stimulus design, from the manifest its *.session.json embeds (§10). The full manifest travels only in those sidecars, and without the design an SSVEP dataset can be decoded but not modelled or rebuilt: target positions, sizes, colours, pattern and reversal, contrast, luminance and trial timing are the design (AGENTS.md §2.2: the stimulus stays describable without the lab's files). So each exported run's _eeg.json carries SSVEPStimulusDesign, regenerated from that manifest key by key, numbers and enums only.

Also not exported: CHANGES (it carries the converter's release date), .bidsignore (nothing non-standard remains to ignore), the converter's *.converter.json marker, and derivatives/ (analysis outputs are out of scope; raise it before extending the export to cover them).

Every file is regenerated, never copied (§6). Nothing is ever read from outside bids_root, and a link anywhere in the tree (a file or a folder) is skipped rather than followed, so it can't pull in content from elsewhere on disk. On Windows that includes a directory junction (mklink /J), which Python 3.11 does not call a symlink: any reparse point that stands for another path is a link here (export._is_link). (push.data_files still checks is_symlink() only; push mirrors to the lab's own storage, not out.)

The destination must be new or empty, and with Also write a .zip so must <destination>.zip: an existing archive is refused, never overwritten. The export is built in a staging folder beside it and moved into place only when complete, so a failure never leaves a half-export that looks whole. Exporting over an earlier export is refused (ExportResult.ok is false and nothing is touched): it would leave behind whatever the earlier one held that this one does not, such as a session whose consent changed since, or sourcedata/ from an export made before this allow-list existed.


3. Why participants.tsv and the rest are filtered

De-identification is enforced on shape, not convention (docs/DATA_GOVERNANCE.md); each deployment binds that to its own board's rules, which here limit participants.tsv to sub-XXX + coarse demographics — no names/DOB/contact/session dates. A row is still a fact about a specific coded participant, so a subject excluded from an export (no eligible session under that purpose) must not have so much as a coarse-demographics row in it. The export keeps only rows whose participant_id has at least one included session.

It also allow-lists columns to a fixed set (participant_id, species, group, age, sex, hand, weight, height, and for future use only age_range and gender; export.participants_columns(purpose)), drops anything else and names each dropped column in ExportResult.refused. Each kept column is checked for its kind (export.PARTICIPANTS_COLUMNS), through the same cell check as every other table:

Column May hold
participant_id sub- plus digits
species homo sapiens
group a code (one word, at most 64 characters)
age a number of years, or a range (18-25, 89+), which the governance rules prefer
sex M/F/O or male/female/other, any case
hand L/R/A or left/right/ambidextrous, any case
weight, height a number
age_range (future use only) one of the background questionnaire's own age-bin values (17-24 … 65+, or prefer_not_to_say), exactly as written
gender (future use only) one of its own gender values (woman, man, non_binary, prefer_not_to_say), exactly as written

Anything else in a kept column (an age of 23 (born in …), a note typed into hand, a gender typed in by hand, an option's display label instead of its value) is written n/a and named, without the value. group goes out as declared (#269), and the declared value is checked the same way. The two background-questionnaire columns take only the shipped instrument's values (ssvep.runtime.questionnaire.AGE_BINS/GENDER_OPTIONS): under future use, a site that edited the instrument's options gets its own values written n/a and named, rather than exported unchecked; under a public share the whole column is withheld before any cell is examined, so an off-list value there is neither written nor named — only the column-level refusal (below) appears.

age_range and gender are exported for future use only, never in a public share. They come from the background questionnaire (docs/BIDS.md, Demographics). In a small sample, an age bin holding one or two people can single them out together with gender, and a public share is open to anyone, with no agreement about what they may do with it. A future-use export stays with the team. So a public share drops both columns and names each one present in ExportResult.refused, with the reason public share: not exported (future use only), so the omission is visible in the Analyze tab rather than silent. Merging bins by hand does not change this: the rule is about the columns, not the bins. participants.json is regenerated for exactly the columns exported. The converter documents age_range and gender there even when nobody has answered the questionnaire and the columns are absent, so an entry for either column that isn't exported is dropped without a refusal of its own (the column's refusal, when there was one, already says so). Any other entry for a column that isn't exported is dropped and named.


4. Using it — the Analyze tab

Export… sits below the Analyze/Open report/Open index row, with a purpose dropdown and two boxes:

  1. Pick Future use (internal reuse) or Public share.
  2. Tick Include stimulus pictures only if every picture this dataset's runs showed may be redistributed (§8). The box applies to one export and clears after it.
  3. Tick Also write a .zip for a reproducible archive beside the folder (§7).
  4. Click Export…. The tool re-scans every session under the BIDS root's sourcedata/ and re-reads each one's data-use stamp right then — not from whatever the tree above happens to have ticked (that selection is for analysing, a different operation).
  5. If nothing is eligible, you're told so (and shown why, per session) and nothing further happens.
  6. Otherwise you see a count of what will be exported and what will be held back (with reasons: the data-use stamp, or a pilot / development session, held back under either purpose), whether pictures will be included, and confirm.
  7. Choose a new or empty destination folder via the file dialog (its default is absolute, ui._paths.export_start_dir).
  8. The export runs on a worker thread. The status label and a summary dialog report what was exported, what was held back, anything the allow-list left out, and any run not converted yet. The left-out list is export.summarize_refused(ExportResult.refused): refusals that differ only in their sub-/ses-/run-/acq- labels are counted on one line, most frequent first, at most 20 lines, so one hand-edited column doesn't bury the refusal that matters.

Nothing here decides when to export or who receives the folder afterwards — that remains a human call, as it is for the push-to-server path. This tool's job ends at "here is the consent-correct, allow-listed dataset, on disk, ready to hand off"; carrying it further is bound by the deployment's residency rules like any other transfer.


5. Compliance notes

No PII added. The export is a reduced regeneration of an already de-identified tree; it invents nothing and reads nothing from outside bids_root.

A "Public share" export is a step toward a release, not itself a publication. The destination folder stays subject to the deployment's data-residency rule until a release decision outside this tool actually publishes it.

No new dependency. The export is stdlib (csv, json, hashlib, zipfile, pathlib), plus Pillow, which image stimuli already require, to read a picture's EXIF when pictures are included (§8).

Testing. tests/test_export.py covers the data-use gate as a pure function (eligible_sessions/excluded_sessions/consent_gate_reason, every matrix cell under both purposes, legacy and unreadable stamps refused) and the export: an unknown file is refused and an unknown key or column dropped; nothing from sourcedata/ under either purpose; the same bytes for both purposes; planted free text and the operator's identity never reach the output; pictures only with the opt-in, and stim_file → n/a otherwise; no real date anywhere; fixed file times; byte-identical archives; the ratings in phenotype/; and a real conversion exporting with nothing refused (bar the public-share licence, §6) and reading back with MNE-BIDS. A real conversion with the background questionnaire answered carries age_range/gender and their documentation under future use, and drops and names both under a public share; a value off the instrument's list never leaves under either purpose; and a dataset without those columns is unaffected (§3). Every leak an adversarial review found is a test that fails if it comes back: stamps that cannot be read (push too, tests/test_push.py), free text in an allowed participants.tsv column or label column, a sentence in a BrainVision header or marker, a junction, a quoted compact date past the field checks, date forms beyond ISO, JSON numbers that read as dates, a .eeg that doesn't match its header, a manifest's own description, a BIDS session with no sourcedata/, and a public share's restrictive licence. Pilot / development sessions (§1): every one is held back and listed, under either purpose, and a pilot-only participant's row is dropped and named, under either purpose too; a mixed participant, a pilot-only one and a session with one pilot recording of two; only a JSON true declares; a pilot session whose XDF is gone is still held back; no sessions table is ever written, a lab table with a column a person added is named while one a held-back session already explains is not, and the converter's own table is left behind silently; a session declared pilot or test afterwards, or with unreadable declarations, is held back under both purposes while the participant's other session is exported (#400); SSVEPConverterPilotGroups never leaving; a real pilot conversion held back whole under both purposes; and a study-only export unchanged, byte for byte, pinned against the pre-gate export, with or without the converter's SSVEPPurposes key and sessions table. An opted-in picture carrying each kind of embedded metadata (EXIF dates, GPS, names, serials, software, descriptions, a thumbnail; XMP; IPTC; a JPEG comment; PNG text and tIME; unknown chunks and segments; bytes after the image; a format nobody walks) is refused and named without its value, and clean PNGs and JPEGs pass (§8). The stimulus design round-trips a synthetic manifest's design facts; its free text, paths, hashes and dates never leave; a value of the wrong kind is dropped and named; the bytes are the same under both purposes; every key and enum of the manifest schema is decided about, so a new manifest field fails a test; and, when the official validator is installed, the exported _eeg.json passes its sidecar rules (§10). tests/test_analyze_ui.py covers the panel wiring without starting the worker thread.


6. The allow-list, and how it fails closed

One allow-list (export.ALLOW_LIST, plus the column and key tables beside it) serves both purposes, so there is one code path and it is the one that is tested. A public share makes two omissions a future-use export does not, and names both: the background questionnaire's participants.tsv columns (§3), and a licence that says the data are not for redistribution (below). Each line names a file pattern under sub-XXX/ses-YYY/ and how that file is regenerated:

File Regenerated from
eeg/*_eeg.vhdr, *_eeg.vmrk Section by section and key by key: only the listed sections and keys, Ch<n>= / Mk<n>= lines, and the first line. ; comments and the [Comment] section are dropped; DataFile/MarkerFile must name the run's own files. Every header value is a code or a comma-separated list of codes (Oz,,0.1,µV), or the run is refused, since a header line can't be dropped. A marker must be Stimulus, Response, New Segment or a sync type — a Comment marker (free text) is dropped — its description must be a code (pybv's S 1, or one word), and its date is kept only if it is shifted.
eeg/*_eeg.eeg Its bytes, read and written anew. The one file not parsed: BrainVision's binary file holds nothing but samples. It must still be what its header says: DataFormat=BINARY, and a size that is a whole number of NumberOfChannels × BinaryFormat samples. Anything else (text under that name, a truncated recording) refuses the run.
eeg/*_eeg.json export.EEG_JSON_KEYS: mne-bids' keys plus the converter's. TaskName and SSVEPRunName must be codes. TaskDescription and Instructions leave only as the converter's generated wording (export.GENERATED_TASK_DESCRIPTION/GENERATED_INSTRUCTIONS, pinned to ssvep.io.bids by a test): a run manifest's own run.description is prose its author typed, and it is dropped and named. SSVEPFlickerRisk, the run's flicker-risk grade the converter carries from the sidecar's stamp (#286 P5), leaves only as one of low, moderate, elevated, high; the factors and the lock-time acknowledgement stay in sourcedata. The export adds SSVEPStimulusDesign, regenerated from the run's manifest through export.STIMULUS_DESIGN_KEYS (§10).
eeg/*_channels.tsv export.CHANNELS_COLUMNS. status_description is BIDS free text and never leaves; description must be one of mne-bids' fixed strings (export.CHANNEL_DESCRIPTIONS).
eeg/*_events.tsv export.EVENTS_COLUMNS; numeric columns must be numbers, label columns (trial_type, value, condition, target_id) codes, stim_file a <sha256>/<name> path (§8).
eeg/*_events.json StimulusPresentation (its listed keys) and a stim_file description the export writes.
eeg/*_electrodes.tsv, *_electrodes.json, *_coordsystem.json Their listed columns and keys.
*_scans.tsv Rebuilt from the runs actually exported: filename, and acq_time only when it is shifted.
sub-XXX_sessions.tsv / .json Never written (§1, #378): its session_type records each session's purpose on the lab's side (#286 P3d), and an export carries research sessions only. The converter writes the table for every converted participant; one holding only its own columns (session_id, session_type) is its bookkeeping and is left behind without a word. A table with any other column holds something a person added, and is named in ExportResult.refused unless one of the participant's pilot sessions is already named in ExportResult.excluded.

Fail closed means: a file matching no line is refused — not exported, and named in ExportResult.refused. So is anything directly under sub-XXX/ that isn't a session folder or the sessions table (above). An unknown key, column, section or marker type is dropped and named there too, as is a value that can't leave (a real date, a control character, a non-number in a numeric column, a sentence where a code belongs).

A code, not a sentence. A listed field is checked for its kind, not only its name. A label cell, a marker description, a BrainVision header value, TaskName and SSVEPRunName must be a code: 1–64 characters with no whitespace (export._CODE). Everything the converter writes into one is (stim_on, 36.0, main, S0, µV); an operator's note typed into one is not, and it doesn't leave. A run missing its header, markers, samples, _eeg.json or _channels.tsv, or whose files can't be read, is refused whole, because part of a recording is not a recording. A run with no sourcedata sidecar is refused too: that sidecar is the only evidence of its data use. Nobody has to have thought of a leak for it to be stopped; somebody has to have listed a field for it to go.

When the converter starts writing something new, the export leaves it out until someone adds it here, and test_a_real_conversion_exports_with_nothing_refused_and_reads_back fails, which is the prompt to decide.

Free text never leaves: no free-text questionnaire item, no operator or session note, no operator identity (in the run sidecar, a questionnaire's provenance or a flag acknowledgement), no channel status_description, no BrainVision comment, and no run manifest's own description.

What dataset_description.json carries, and the public-share licence. It carries the standard BIDS fields the site profile renders, among them Authors, Funding, EthicsApprovals and, in _eeg.json, InstitutionName/InstitutionAddress. That is ordinary BIDS, but it means an export names the deployment's institution and ethics reference, which this repository itself never publishes; decide that before a public release, like the rest of it. Under Public share, a License or HowToAcknowledge that says the data are not for redistribution (the site profile's defaults for a lab's own copy: site.DEFAULT_DATA_LICENSE, and the governance statement) is dropped and named, because a public share whose own metadata forbids sharing contradicts itself. Declare the licence to publish under in the site profile (governance.data_license, docs/SITE_PROFILE.md) before a public share. Future use keeps both as written.


7. Dates and file times

Nothing exported carries a real date. Dates are shifted when they are written (docs/DATA_GOVERNANCE.md), and the export checks again: every string it writes is scanned (export.real_date_in) for a valid date after 1925, in ISO, day- or month-first, compact (20310517, as BrainVision and file names write it), spaced (2031 05 17), month-name (17 May 2031, 17-May-2031, May 17, 2031) or ctime (Wed May 17 13:04:05 2031) form. A two-digit year (05/17/31) is deliberately not read as a date: it is as likely a ratio or a version, and nothing the toolbox writes uses one. Every JSON number is checked too: one that reads as a compact date, or as an epoch time from 2000 to 2100 (seconds or milliseconds), is dropped and named. A shifted date (year ≤ 1925) passes. A real one is dropped, acq_time falls back to n/a, and a last scan over every written file refuses the whole export if one slipped through anyway. That scan reads a quoted number ("20310517" in a JSON sidecar) as the text it is, and skips only a number standing alone, such as a sample index in a table.

The one date-shaped value let through is a g.tec Unicorn's DeviceSerialNumber, UN-YYYY.MM.DD: the day the amplifier was made, not a day anyone was recorded. It passes only as that key's whole value.

Every exported file and folder gets the same modified time, export.FIXED_MTIME (1980-01-01T00:00:00Z, the earliest a zip entry can hold). A real modified time beside a shifted timestamp in the same file would give the shift away, and with it every date in the session. Nothing is copied with shutil.copy2, which would keep the source's time. One time is not fixed: on NTFS each exported file and folder has a creation time, which is when the export ran (the time of the hand-off, not of any recording). The .zip stores no creation time.

archive=True (the Also write a .zip box) writes <destination>.zip beside the folder with export.write_archive: entries sorted by path, each stamped 1980-01-01 00:00, fixed permissions, the same compression, no folder entries. Its bytes depend only on the files' paths and contents, so two exports of the same data give byte-identical archives, whenever and wherever they were made.

What still bounds a date, stated so nobody claims more: SoftwareVersions and GeneratedBy name the toolbox version, and commit dates are public, so an export bounds when it was recorded to within days or weeks. It can't recover a date.


8. Stimulus pictures

Pictures ship only on request, per export (include_stimuli=True, the Include stimulus pictures box). Whether a picture set may be redistributed — third-party copyright, a photograph of a person — is a property of the study, not of the purpose, so it is an explicit choice for each export rather than a default for either purpose.

  • With it, the export writes each picture an exported run's events.tsv names (stimuli/<sha256>/<name>, from the dataset-root stimuli/ the converter fills), only if its bytes hash to the sha256 in its own path. A picture only an excluded session showed is not exported.
  • Without it, or for a picture that can't be verified, the exported stim_file is n/a, so the dataset never names a file it does not contain (the BIDS validator reports that as STIMULUS_FILE_MISSING, an error). The run's _events.json says so.
  • ExportResult.stimuli_not_exported counts the distinct pictures the exported events name that the export does not contain; stimuli_exported counts those it does.
  • A shipped picture is its exact bytes, since its SHA-256 is its name. So a picture that carries embedded metadata is refused, never stripped — here: stripping would change the bytes and break the pin its stim_file rests on, so export itself only ever refuses and names what it found, never rewrites. The refusal is named in ExportResult.refused with the kinds of metadata found (EXIF DateTimeOriginal, EXIF GPS (GPSLatitude), XMP, PNG text 'Author'), never their values; its stim_file is n/a, and it counts in stimuli_not_exported. Stripping happens earlier, at bundle time (docs/IMAGE_STIMULI.md, #355): Build run's picture button re-encodes a freshly chosen file to a clean PNG — no EXIF, no colour profile — before it is ever bundled or pinned, so a picture added through the normal workflow does not carry metadata to refuse in the first place. This refusal is the backstop for a bundle built some other way: a manifest or protocol hand-edited outside Build run, or an older bundle from before #355. For one of those, strip the metadata from the picture (for example exiftool -all=) and bundle the stripped file into the design again: the stripped file is a different file, with its own SHA-256.
  • What counts as metadata (export.picture_metadata). PNG and JPEG, the formats pictures are normally bundled in, are walked chunk by chunk and segment by segment against an allow-list of what holds pixels, colour and layout, and each allowed chunk or segment must be exactly as long as its fields, so none can carry anything else: a PNG sRGB, pHYs, tRNS or sBIT with extra bytes, a second sRGB, a second run of IDAT chunks, bytes after the image data's compressed stream ends or more image data than the picture's size, a JFIF or Adobe segment with extra bytes, and a JPEG table or frame segment longer than its tables are all refused. EXIF is read tag by tag with Pillow, and a Pillow warning counts as an error, so a block Pillow can't fully read is refused rather than read as empty. Only the image-structure tags pass (size, layout, orientation, resolution, colour space, the EXIF version), and only holding what they hold (a number, or a version code): a structural tag number holding text is refused. The Interop directory is read too, and passes only its index (R98) and version. An EXIF block that is empty, chains to a second directory (a thumbnail) or holds bytes no tag accounts for is refused. Anything else is refused, which covers a capture or modification date, GPS, Artist, Copyright, CameraOwnerName, a body or lens serial, Software, ImageDescription, UserComment and a maker note without anyone having to list them. So is an XMP packet, IPTC or other Photoshop resources, a JPEG comment, a PNG text chunk (tEXt/zTXt/iTXt) or tIME, an embedded thumbnail (EXIF or JFIF: a thumbnail can be the uncropped original), an animated PNG's frames (the toolbox shows one picture), any other chunk or segment, and anything after the end of the image. An embedded colour profile is refused too (PNG iCCP, JPEG ICC_PROFILE): the toolbox never applies one (it is not colour-managed: ssvep.stim.images), so the profile describes nothing that was shown, and a profile carries its maker, a free-text description and copyright line (a monitor profile is often named after its owner's machine) and a creation date. sRGB, gAMA and cHRM carry colour intent without free text and still pass. Any other format (GIF, WebP, TIFF, …) is refused as one the export can't account for byte by byte; convert it to PNG — which is exactly what Build run's picture button already does to any of those formats before bundling (#355), so this case is now mostly the same hand-edited-bundle backstop as the rest of this list.
  • A refusal repeats only names from fixed lists, never what the file's writer chose: a registered PNG text keyword (Title, Author, Description, Copyright, Creation Time, Software, Disclaimer, Warning, Source, Comment, ImageMagick's Raw profile type …), a registered PNG chunk type, or Pillow's EXIF tag names. A PNG keyword is up to 79 characters of its writer's choosing, so any other keyword is reported as "a PNG text chunk with an unregistered keyword" and any other chunk type as "a PNG chunk of an unregistered type".
  • In practice, a photograph straight from a phone or camera almost always carries EXIF (a date, usually GPS) and often a colour profile. Picked through Build run's picture button it is re-encoded and stripped automatically before it is bundled (#355), so this no longer blocks a normal design — it remains the reason to re-bundle a picture that reached its bundle some other way (a hand-edited manifest or protocol, or a bundle from before #355): strip it (exiftool -all=, or re-save it as PNG without a profile) and bundle the stripped file into the design again. A picture already recorded with keeps its SHA-256 either way, so the only cure for one already shown is a new bundle and a new recording.
  • Ticking the box is still a statement about what the pictures show (copyright, a photograph of a person). Metadata is only the part a machine can check.

9. Questionnaire ratings: phenotype/

The rating-scale responses a shared dataset promises come from sourcedata/sub-XXX/ses-YYY/beh/*_beh.json, which are read but never exported as written. The export writes phenotype/<task>.tsv, one row per administration, with participant_id, session_id, administration, group, after_run_index, state (complete or declined) and acq_time (the shifted finish time, or n/a), then one column per rating item. phenotype/<task>.json is its data dictionary: each item's wording and scale labels, from the instrument embedded in the response file.

  • Ratings in, free text never. A free-text item is not read into the output at all, and neither are the raised flags, the operator's flag acknowledgement or the questionnaire's provenance.
  • A rating that isn't an integer on its item's scale is written n/a; a declined administration's ratings are all n/a.
  • An administration naming a protocol the session's recordings don't is refused: that is another protocol's answers under this participant code (#279).
  • When an instrument's wording differs between administrations in the exported set, each wording gets its own table (<task>v1, <task>v2, …), so no column mixes two questions.

10. The stimulus design: SSVEPStimulusDesign

Each exported run's _eeg.json carries SSVEPStimulusDesign: the stimulus design the run was recorded with, regenerated from the manifest its sourcedata/ sidecar embeds (export.stimulus_design). A legacy manifest is canonicalized first (ssvep.io.manifest.canonicalize_manifest), so it reads as it is read everywhere else. A run whose sidecar embeds no manifest gets no key, and the exported README says whether all, some or none of its runs carry one.

Numbers, booleans, the schema's own enums and codes only, key by key (export.STIMULUS_DESIGN_KEYS). Keys and units are the run-manifest schema's (schemas/run_manifest.schema.json), so a reader can look each one up there:

Section Exported
top level manifest_version, paradigm, and n_targets (the number of targets the run showed, added by the export)
display refresh_hz, measured_refresh_hz, resolution_px, gamma, background_rgb, luminance_cd_m2, units, viewing_distance_cm, screen_diagonal_cm
stimuli[] id (an identifier, as target_id in _events.tsv; see below), position_xy, size, shape, color, cue_color, pattern, pattern_reversal, spatial_freq_cpd; flicker: type, freq_hz, rendered_freq_hz, phase, contrast, duty; a picture's image: source_px, fit, mean_luminance, rms_contrast, has_alpha
design conditions[]: id (an identifier or a factorial id, as condition in _events.tsv), stimulus_freqs_hz; blocks_per_condition, trials_per_block, method; timing: cue_s, stim_s, iti_s, onset_offset_s; resting: block_s, n_blocks, start, high_tone_hz, low_tone_hz, tone_ms, lead_in_s, settle_s, fixation_cross, band_hz; early_stop: statistic, method, min_window_s, step_s, confidence_margin, calibration.source

Never exported, each listed as such rather than merely left unlisted, so a refusal list names only what nobody has decided about: run (its name, task, prose description and builder version), a condition's label and params, an early-stopping threshold's calibration.basis (free text its author typed; calibration.source says whether there was one), a factorial design's factors and planned trial_order (free-form keys; the order that actually ran is in _events.tsv, and how it was chosen, design.order's scheme and seed, is exported), a picture's path and sha256 (a path into the lab's bundle, and a hash that would tie the export to the lab's picture files; a shipped picture is named by stim_file instead, §8), acquisition (amplifier, headset and montage: the channel and electrode files carry what a reader needs), and markers (the stream's name and a schema path). No fingerprint is exported, neither the manifest's nor the protocol's, and the manifest's can't be recomputed from what is, since run and acquisition are part of it.

Target and condition ids are identifiers, or the run is not exported. A target's id and a condition's id are named by whoever designed the run, and the same id is every target_id or condition cell of the run's _events.tsv. So at export (only: design-time validation and the manifests are unchanged) each must be an ASCII letter followed by ASCII letters, digits or underscores, at most 32 characters, with no run of six or more digits (S0, key_up, main, eyes_open). A condition id may also be the factorial generator's name=level__name=level form (each name an identifier, each level a number, T/F or an identifier). That refuses an email, a URL, a phone or student number, an apostrophe, a space and every non-ASCII or zero-width character. A run whose ids don't fit, in its design or in any target_id/condition cell of its events, is refused whole and named by run and position (stimuli[1]: id, row 3: a target_id), never by the id itself; nothing else of that run is reported, since none of it is exported. Every preset, every tracked example manifest and every id the factorial generator writes fits, and a test holds that. What the rule can't do: a bare JaneDoe is an identifier, because a code cannot tell a name from a word. Relabelling ids to t0…tN/c0…cM in the design and the events alike would close that too, and is a possible follow-up.

It fails closed like everything else. A value of the wrong kind (prose where a number or code belongs, a number that reads as a date or an epoch time, a value outside the schema's enum, a vector with a non-number in it) is dropped and named, and so is a manifest key nobody listed. A test pins the table above to the schema: every key the schema defines (including one defined only inside a conditional allOf/if/then/else block) is either exported with its kind or never exported, every enum is the schema's own, and every vector's length bounds are the schema's (a vector of the wrong length is dropped and named). A key under a NEVER section (run, acquisition, markers) is covered by that section, not listed on its own. An object left empty once its NEVER keys are gone is left out, and n_targets is written only when every target was exported. A new manifest field therefore fails that test until someone decides whether it is design.

Why here, and not a file of its own. BIDS lets a sidecar carry keys the specification does not define, and the converter already writes its SSVEP facts that way (SSVEPStimulusFrequenciesHz, SSVEPRunName), so one more extension key keeps the dataset valid with nothing to add to a .bidsignore (the export writes none). A dataset-level task-<label>_stim-design.json is not a BIDS file name, so the validator would reject it. Its per-task inheritance would be wrong anyway: two runs of one task can show different designs, and the manifest is per run. The run's _eeg.json is the file that describes the recording, and it already carries the target frequencies. When the official validator is installed, a test runs it on an export and asserts no error in any _eeg.json.