Sensory perception relies on fitness-maximizing codes

Sensory information encoded by humans and other organisms is generally presumed to be as accurate as their biological limitations allow. However, perhaps counterintuitively, accurate sensory representations may not necessarily maximize the organism’s chances of survival. To test this hypothesis, we developed a unified normative framework for fitness-maximizing encoding by combining theoretical insights from neuroscience, computer science, and economics. Behavioural experiments in humans revealed that sensory encoding strategies are flexibly adapted to promote fitness maximization, a result confirmed by deep neural networks with information capacity constraints trained to solve the same task as humans. Moreover, human functional MRI data revealed that novel behavioural goals that rely on object perception induce efficient stimulus representations in early sensory structures. These results suggest that fitness-maximizing rules imposed by the environment are applied at early stages of sensory processing in humans and machines.


Reporting Summary
Nature Research wishes to improve the reproducibility of the work that we publish. This form provides structure for consistency and transparency in reporting. For further information on Nature Research policies, see our Editorial Policies and the Editorial Policy Checklist.

Statistics
For all statistical analyses, confirm that the following items are present in the figure legend, table legend, main text, or Methods section.
n/a Confirmed The exact sample size (n) for each experimental group/condition, given as a discrete number and unit of measurement A statement on whether measurements were taken from distinct samples or whether the same sample was measured repeatedly The statistical test(s) used AND whether they are one-or two-sided Only common tests should be described solely by name; describe more complex techniques in the Methods section.
A description of all covariates tested A description of any assumptions or corrections, such as tests of normality and adjustment for multiple comparisons A full description of the statistical parameters including central tendency (e.g. means) or other basic estimates (e.g. regression coefficient) AND variation (e.g. standard deviation) or associated estimates of uncertainty (e.g. confidence intervals) For null hypothesis testing, the test statistic (e.g. F, t, r) with confidence intervals, effect sizes, degrees of freedom and P value noted

Give P values as exact values whenever suitable.
For Bayesian analysis, information on the choice of priors and Markov chain Monte Carlo settings For hierarchical and complex designs, identification of the appropriate level for tests and full reporting of outcomes Estimates of effect sizes (e.g. Cohen's d, Pearson's r), indicating how they were calculated Our web collection on statistics for biologists contains articles on many of the points above.

Software and code
Policy information about availability of computer code Data collection Matlab 2019; Psychtoolbox-3 Data analysis R: A language and environment for statistical computing (version 3.6.3); R package brms (version 2.13.5); JAGS 4.3 For manuscripts utilizing custom algorithms or software that are central to the research but not yet described in published literature, software must be made available to editors and reviewers. We strongly encourage code deposition in a community repository (e.g. GitHub). See the Nature Research guidelines for submitting code & software for further information.

Data
Policy information about availability of data All manuscripts must include a data availability statement. This statement should provide the following information, where applicable: -Accession codes, unique identifiers, or web links for publicly available datasets -A list of figures that have associated raw data -A description of any restrictions on data availability Data availability statement is provided in the submitted manuscript. The Lateral Occipital Cortex (LOC) mask was obtained from the fMRI meta-analysis tool Neurosynth (neurosynth.org) V1-V3 masks were extracted from the Julich-Brain Cytoarchitectonic Atlas (julich-brain-atlas.de)

nature research | reporting summary
April 2020 Field-specific reporting Please select the one below that is the best fit for your research. If you are not sure, read the appropriate sections before making your selection.

Blinding
Describe whether the investigators were blinded to group allocation during data collection and/or analysis. If blinding was not possible, describe why OR explain why blinding was not relevant to your study.

Behavioural & social sciences study design
All studies must disclose on these points even when the disclosure is negative.

Study description
We designed an experiment to test if human early visual areas adapt in a manner predicted by the theory of fitness maximization. The data is quantitative experimental, allowing to test direct qualitative predictions of the theory (see Figure 3).

Research sample
The sample consisted of young healthy volunteers (age range: 18-40) who were mostly University students/employees. Given that we wanted to test the hypothesis that early visual areas adapt to fitness-maximizing codes in the healthy brain, none of the participants suffered from any neurological or psychological disorder or took medication that interfered with participation in our study.

Sampling strategy
The sampling strategy was random. The sample size in Experiment 1 was determined such that the final sample size after exclusion criteria (see below) consisted of at least 25 participants each with 200-400 successful trials per experiment. This sample size typically allows testing well-defined theories of behaviour in standard laboratory settings (e.g., Polania et al., 2019 Nature Neuroscience). The replication/extension sample size in Experiment 2 was chosen for the same reasons.

Data collection
Stimuli were generated with Matlab, using the Psychtoolbox and displayed on a screen using the following experimental protocol. Participants were sat one meter from the screen. The angle of the head was kept stable with a chin rest. The height of the chin rest was adjusted to position the center of the screen at the height of the eyes. As orientation stimuli, we used oriented Gabor patches, presented on a grey background.
Eye-tracking data was acquired using an ST Research Eyelink 1000 eye-tracking system. Gaze position was sampled at 500 Hz.
no one else was present besides the participants(s) and the researcher. The researcher was not blinded to experimental condition and/or the study hypothesis.

Timing
Data collected from the end of 2018 until March 2022

Data exclusions
Participants who failed to follow the eye fixation instructions on more than 25\% of trials were excluded from the data analysis. We measured the performance of participants in the training tasks and excluded eleven that were unable to perform the task at the easiest difficulty level. Additionally, we had to exclude three participants due to technical problems with the data collection. Thus, the final sample comprised n=86 participants (n=25 in Experiment 1, n=61 in Experiment 2).

Non-participation
Participants were recruited according to the following inclusion cireterion: None of the participants suffered from any neurological or psychological disorder or took medication that interfered with participation in our study.

Sampling strategy
Note the sampling procedure. Describe the statistical methods that were used to predetermine sample size OR if no sample-size calculation was performed, describe how sample sizes were chosen and provide a rationale for why these sample sizes are sufficient.

Data collection
Describe the data collection procedure, including who recorded the data and how.

Location
State the location of the sampling or experiment, providing relevant parameters (e.g. latitude and longitude, elevation, water depth).

Access & import/export Describe the efforts you have made to access habitats and to collect and import/export your samples in a responsible manner and in
compliance with local, national and international laws, noting any permits that were obtained (give the name of the issuing authority, the date of issue, and any identifying information).

Disturbance
Describe any disturbance caused by the study and how it was minimized.

Reporting for specific materials, systems and methods
We require information from authors about some types of materials, experimental systems and methods used in many studies. Here, indicate whether each material, system or method listed is relevant to your study. If you are not sure if a list item applies to your research, read the appropriate section before selecting a response.

Eukaryotic cell lines Policy information about cell lines
Cell line source(s) State the source of each cell line used.

Authentication
Describe the authentication procedures for each cell line used OR declare that none of the cell lines used were authenticated.

Mycoplasma contamination
Confirm that all cell lines tested negative for mycoplasma contamination OR describe the results of the testing for mycoplasma contamination OR declare that the cell lines were not tested for mycoplasma contamination.

Commonly misidentified lines (See ICLAC register)
Name any commonly misidentified cell lines used in the study and provide a rationale for their use.

Specimen provenance
Provide provenance information for specimens and describe permits that were obtained for the work (including the name of the issuing authority, the date of issue, and any identifying information).

Specimen deposition
Indicate where the specimens have been deposited to permit free access by other researchers.

Dating methods
If new dates are provided, describe how they were obtained (e.g. collection, storage, sample pretreatment and measurement), where they were obtained (i.e. lab name), the calibration program and the protocol for quality assurance OR state that no new dates are provided.
Tick this box to confirm that the raw and calibrated dates are available in the paper or in Supplementary Information.

Ethics oversight
Identify the organization(s) that approved or provided guidance on the study protocol, OR state that no ethical approval or guidance was required and explain why not.
Note that full information on the approval of the study protocol must also be provided in the manuscript.

Animals and other organisms
Policy information about studies involving animals; ARRIVE guidelines recommended for reporting animal research

Laboratory animals
For laboratory animals, report species, strain, sex and age OR state that the study did not involve laboratory animals.

Wild animals
Provide details on animals observed in or captured in the field; report species, sex and age where possible.

Ethics oversight
Identify the organization(s) that approved or provided guidance on the study protocol, OR state that no ethical approval or guidance was required and explain why not.
Note that full information on the approval of the study protocol must also be provided in the manuscript.

Human research participants
Policy information about studies involving human research participants

Population characteristics
See above

Recruitment
Participants were recruited via the online system recruitment platform of the University of Zurich. There might be slight selection bias in our sample given that participants were recruited online and required access to a computer or mobile phone. However, given that we here we study basic sensory system mechanisms, we believe that this selection had no impact in the interpretation and conclusions derived in our work

Ethics oversight
The experiments conformed to the Declaration of Helsinki and the experimental protocol was approved by the Ethics Committee of the Canton of Zurich.
Note that full information on the approval of the study protocol must also be provided in the manuscript.

nature research | reporting summary
April 2020 Clinical data Policy information about clinical studies All manuscripts should comply with the ICMJE guidelines for publication of clinical research and a completed CONSORT checklist must be included with all submissions.
Clinical trial registration Provide the trial registration number from ClinicalTrials.gov or an equivalent agency.

Study protocol
Note where the full trial protocol can be accessed OR if not available, explain why.

Data collection
Describe the settings and locales of data collection, noting the time periods of recruitment and data collection.

Outcomes
Describe how you pre-defined primary and secondary outcome measures and how you assessed these measures.

Dual use research of concern
Policy information about dual use research of concern

Hazards
Could the accidental, deliberate or reckless misuse of agents or technologies generated in the work, or the application of information presented in the manuscript, pose a threat to:

Experiments of concern
Does the work involve any of these experiments of concern: No Yes Confirm that both raw and final processed data have been deposited in a public database such as GEO.
Confirm that you have deposited or provided access to graph files (e.g. BED files) for the called peaks.

Data access links
May remain private before publication.
For "Initial submission" or "Revised version" documents, provide reviewer access links. For your "Final submission" document, provide a link to the deposited data.

Files in database submission
Provide a list of all files available in the database submission.
Genome browser session (e.g. UCSC) Provide a link to an anonymized genome browser session for "Initial submission" and "Revised version" documents only, to enable peer review. Write "no longer applicable" for "Final submission" documents.