Skip to content

Image stimuli — pictures as flicker targets (#261 of #255)

A target can flicker a picture instead of a solid colour or a generated checkerboard/grating — an icon on a speller key, a game affordance, or any image you provide. This page is the practical guide: how to add one in Build run, what gets pinned into the manifest and why, where the picture file itself lives, and the caveats worth knowing before you record with one.

A target here shows one picture for the whole run. For a stream that shows a new picture every cycle (fast periodic visual stimulation, words among letter strings and the like), see FPVS: it is a run type of its own, built in Python rather than in Build run, on the same pinned pictures and asset bundles described below.

The full mechanics — the manifest's image block, how a protocol bundles pictures from the runs it embeds, and what a locked protocol refuses — are in Protocols and sessions and Protocols and sessions § Picking and previewing. This page does not repeat those sections; it is the front door to them.

Adding a picture target in Build run

A target row's pattern cell is a dropdown (solid / checkerboard / grating) plus a picture button. The button opens a dialog to choose a picture file (any format Pillow can decode — PNG, JPEG, BMP, GIF, WebP, TIFF and the other common raster formats), preview it, and set fit — how the picture fills the target's rectangle. Accepting it re-encodes the file to a clean PNG before bundling it — EXIF orientation applied, then EXIF, any embedded colour profile and any PNG text dropped, pixels/mode/alpha kept unchanged (#355; Caveats below has why) — and that is what gets a hash and a place in the bundle, not the original file. The dialog's mechanics — staging the re-encoded file into the design's own asset bundle, filling in the rest of the pinned facts, and what round-trips byte-for-byte when a target is never touched — are in Protocols and sessions § Picking and previewing; this page only covers what you need to decide.

fit has three options:

  • contain — the whole picture, letterboxed on one axis with the run's background; never distorts or crops.
  • cover — fills the target completely, centre-cropped on one axis.
  • stretch — the whole picture stretched into the whole target; aspect ratio is not preserved (see Caveats below). The Validation panel warns when a stretch fit distorts the aspect ratio by more than roughly 10% — the picker dialog can't raise that warning itself, because it doesn't yet know the target's on-screen size.

The dialog's own preview is a plain scaled copy of the picture; it is not composited against the run's background. The layout preview, drawn from the target's actual on-screen rectangle, does composite the picture over the run's real background.

If you just want to see the feature working without picking a file of your own, load the "Image keys" preset from Build run's preset list — four generated arrow-key pictures on a 30–36 Hz grid, drawn in code so nothing needs sourcing.

What the pinned block records, and why

Choosing a picture stamps a per-target image block into the run manifest (schema v1.4 — a run with no image target still emits the older v1.3, unchanged):

Field What it is Why it's pinned
path Where the picture is, relative to the manifest's own asset bundle Says where to look — but never what is there (see sha256)
sha256 The SHA-256 hash of the picture file's bytes The picture's identity. A run manifest had never referenced anything outside itself before this; pinning the hash means the manifest's fingerprint covers what was shown, not just a filename, and a substituted or corrupted file is refused rather than shown in its place
source_px The picture's native pixel dimensions Describes the source without needing the file present
fit contain / cover / stretch How the picture was sized into the target — needed to reconstruct what was on screen
has_alpha Whether the file carries transparency Affects how it composites onto the background (see Caveats)
mean_luminance Mean luminance of the picture as it would render, in linear light, on this run's background and gamma So the picture's light — not just its identity — is describable without the file. Analysis can reason about what was on screen from the manifest alone
rms_contrast RMS contrast of the same rendering Same reason: a manifest is meant to contain everything analysis would otherwise have to infer, and a picture's contrast is exactly the kind of thing that would otherwise require re-opening the file

mean_luminance and rms_contrast are measured once, at design time, against the run's declared background colour and gamma — if you change either later, Build run re-measures automatically so the pinned numbers never silently describe a display that no longer exists.

Where pictures live

Pictures are not stored inside the manifest or protocol file itself. Saving a design with an image target copies the picture into a bundle directory beside the file you saved:

  • Build run's Save → <run-manifest-stem>.assets/
  • Build Protocol's Save → <protocol-stem>.assets/, holding the pictures for every run the protocol embeds

These bundles are kept out of git. .gitignore excludes every *.assets/ directory, and a compliance test (tests/test_compliance_data.py) independently asserts none is tracked, however it got staged — the same git-side guard the toolbox uses elsewhere, because a picture can be copyrighted or otherwise not yours to redistribute (see Copyright and redistribution below). Only generated, code-drawn examples (like "Image keys") ship with the toolbox; no third-party or sourced picture does.

A picture also travels with the recording it was shown in: the recorder copies each one, keyed by its content hash, into the session's sourcedata/…/stimuli/<sha256>/ folder. Push mirrors that folder to the lab's storage with the rest of the session, under the session's consent gate, and the BIDS converter carries the picture into the dataset's stimuli/ folder with an events.tsv stim_file reference — that is the local BIDS view; see BIDS conversion § Image targets for the layout. A picture is included in an export only when the operator asks for it on that export (Include stimulus pictures), under either export purpose; otherwise the exported stim_file is n/a.

Caveats

A picture's spatial contrast does not lower its photosensitivity rating. The classifier that warns on risky flicker rates temporal contrast — how hard the screen flashes between on and off — and an on-off flicker is full-depth whatever the picture shows: a flat grey photograph flashes exactly like a grey square. An image target's measured spatial contrast (rms_contrast above) is therefore deliberately not fed into that rating; a dim or low-contrast picture is rated exactly as a solid target of the same size and frequency would be, never more leniently.

Expect a weaker SSVEP from a dark or low-contrast picture. Response amplitude is driven by how much the screen's luminance actually swings between the on and off states, and a picture's own mean luminance and contrast are both typically below a plain white or saturated-colour square's. Build run's picker warns on a noticeably dark or nearly-uniform picture for this reason — treat the warning as a signal about signal quality, not just a display nicety.

stretch can visibly distort a picture. Unlike contain and cover, which always preserve the picture's aspect ratio, stretch maps the whole picture into the whole target regardless of shape, so a picture whose aspect differs much from the target's will look stretched or squashed. The Validation panel warns when a stretch fit distorts the aspect ratio by more than roughly 10%, as noted above.

Transparency is composited onto the run's background, in linear light — it is never a window onto whatever is behind the stimulus screen. A picture with an alpha channel (has_alpha: true) has its transparent and partially-transparent pixels flattened onto the target's own background colour before it is ever drawn; pixels the picture's shape mask does not cover (letterboxing, the corners outside a round target) show that same background on every frame.

A picture you add is re-encoded, and that's normally invisible — but worth knowing about. Before #355, bundling a photograph verbatim meant it almost always carried EXIF (capture date, GPS, camera make/model) or an embedded colour profile, and export's own checker refuses to ship either (see Consent-aware export § Stimulus pictures) — so a picture recorded with could never be exported without redoing the design by hand. Build run now re-encodes every freshly picked file to a clean PNG before it is bundled, dropping exactly that metadata and nothing else: the pixels, the colour mode and the alpha channel are kept, EXIF orientation is applied first so the picture is not left sideways, and the toolbox never applied the colour profile to begin with (it is not colour-managed), so dropping it changes nothing on screen. The one visible side effect is size: a JPEG photograph typically gets noticeably larger once re-encoded as a lossless PNG. An animated GIF or WebP (or a multi-frame PNG) is refused outright at this same step — the renderer only ever draws a target's first frame, so bundling an animation would silently lose every frame after it. None of this touches a picture already bundled by an earlier Save: manifests are durable contracts, so only a new pick is ever re-encoded.

On-screen appearance and frame timing under image textures are not yet confirmed on hardware. Solid targets have been measured on real display hardware, and pattern (checkerboard/grating) targets have their own QC manifests and measurement SOP (see the flik-o-meter guide); an image target's frame-exactness and appearance under texture load is the subject of a still-open bench check (#263). Treat an image target as functionally complete but not yet hardware-validated the way solid targets are.

Pictures are left out of an export unless the operator ticks Include stimulus pictures for that one export (see Consent-aware export § Stimulus pictures). Whether a picture set may be redistributed is a property of the study, not of the export purpose, so the toolbox asks every time rather than defaulting either way. Left out, the exported events.tsv names no picture (stim_file is n/a), so the dataset still validates.

That is a mechanical backstop, not a substitute for checking rights yourself: you are responsible for having the rights and any permissions a picture needs, including for an identifiable person it shows — the toolbox does not, and cannot, verify this for you. This applies whether the picture came from your own files, a stock library, or anywhere else.