When Information Is Not Enough: Accuracy-Constrained Thermodynamic Costs of Binary Classification

For a fixed q, maximize A=\sum_iw_is_ik_i under 0\leq k_i\leq1 and \sum_iw_ik_i=q. If s_i>s_j while k_i<1 and k_j>0, transferring an equal amount of selected probability from j to i strictly increases A and preserves q. Repeating such exchanges gives the threshold kernel and (?). Symmetry yields the negative counterpart with the same q, so the maximum absolute value is also A_*(q). The error formula in (?) proves the second claim.

For A>0 in the interior of the channel domain,

The function is even in A, with continuous boundary values. Thus maximizing absolute contrast also maximizes information at fixed q. This proves (1). Encoders of cost q>1/2 can be complemented to encoders of cost 1-q<1/2, preserving both Bayes error and mutual information. Hence no optimum requires q>1/2, and (1) follows.

For (1), the binary Fano inequality gives I(Y;M)\geq\ln2-h_2(\epsilon) for any task-feasible encoder. A symmetric Bayes rule, with a fair coin on posterior ties, induces a binary symmetric channel with crossover probability r. Flipping its decision independently with probability (\epsilon-r)/(1-2r) produces a binary symmetric channel of crossover probability \epsilon, which achieves equality. This encoder factors through D, so the lower bound is achievable within the observation constraint. The fixed-cost information boundary now gives (1). Compactness of the finite-dimensional kernel cube and continuity establish attainment.

Let p_+=P(s_D>0)\leq1/2. Since 0<\delta<S, the threshold encoder reaches contrast \delta before exhausting all positive scores. Therefore 0<q_T<p_+\leq1/2, and its Bayes error is exactly \epsilon.

We show that Fano is strict for this encoder. If equality held in H(Y\mid M)\leqh_2(\epsilon), the binary error indicator E for a Bayes decoder would be independent of M. Since the error is less than 1/2, the decoder is nonconstant. Uniformity of Y would then give

forcing P(g(M)=1)=1/2, hence q_T=1/2, a contradiction. Thus \Psi(q_T)>R_D^Y(\epsilon). Continuity, \Psi(0)=0<R_D^Y(\epsilon), and the intermediate value theorem yield a feasible information cost strictly below q_T.

For k_d=0, keep both rates zero. For k_d>0, prescribe the conditional memory probability

Define \rho(t)=\min\{\Lambda t,c\} and rates

The two rates are finite and continuous across t_0=c/\Lambda. Direct substitution gives

so the master equation is satisfied exactly from x_d(0)=0. The backward rate is bounded by \Lambda. The forward rate obeys

The log ratio of the forward and backward rates is finite on every active sector, including t=0; hence the rates can be assigned a finite instantaneous energy difference under local detailed balance.

Writing J_d=w_dk_d/\tau, the joint fluxes are

Every active sector has the same flux force

Consequently \mathcal M=(q/\tau)\int_0^\tau f(t)^{-1}\,\mathrm{d} t\leq q/F=\mathsf{B}. The force diverges logarithmically as t\downarrow0, due to the pure initial distribution, but the rates do not diverge and the entropy integral is finite. Since t_0<\tau,

The last equality follows from \int_0^c\ln(1+1/z)\mathrm{d} z=(c+1)\ln(c+1)-c\ln c. Combining this explicit upper bound with (?) proves the claim.

Every task endpoint has q\geq q_T, giving C_{T,\Lambda}\geq q_T^2/\mathsf{B}. Theorem 5 supplies an actual information-feasible protocol with cost at most the right-hand side of (5) at q_I, as well as a finite-cost task-feasible protocol at q_T. Subtract the information upper bound from the task lower bound. This compares achievable costs with a universal lower bound, not two lower bounds.

Theorem 3 proves that the task-optimal threshold encoder k^T has information strictly above R_D^Y(\epsilon). Replace it by k_d=\alpha k_d^T for 0<\alpha<1 sufficiently close to one. Continuity preserves information strictly above the threshold, while q=\alpha q_T<q_T and \max k_d\leq\alpha<1. Thus the information witness required by the theorem always exists.

To ensure task feasibility with finite rates, put \delta=1/2-\epsilon and let k^B be a Bayes rule with contrast S, randomizing ties symmetrically. Choose \beta\in(\delta/S,1) and set k_d=\beta k_d^B. This has contrast \beta S>\delta, hence error strictly below \epsilon, and all encoder probabilities are below one. Theorem 5 realizes both witnesses at sufficiently large finite caps.

Every task-feasible protocol has q\geq q_T, and hence entropy production at least q_T^2/\mathsf{B}. The explicit information witness has entropy production given by (5). Subtracting gives (8). All rate and strictness thresholds are finite since both witnesses have maximum write probability below one and q<q_T. This also establishes a common \Lambda_0 valid for every larger cap.

The conditional probability satisfies \dot x_d\leq\Lambda(1-x_d) and x_d(0)=0, so k_d\leq1-e^{-\Lambda\tau}=:\gamma. By symmetry,

Thus e=1/2-|A|\geq1/2-\gamma S=r+e^{-\Lambda\tau}(1/2-r).