Research Reality Cards

The study successfully identifies and quantifies the divergence between salience and selection framing in French news headlines, revealing unexplained variance in outlet-level framing.

The research analyzes 902,111 deduplicated headlines from 25 French outlets, marking it as the largest framing-focused audit to date.

The study's reliance on majority-vote resolution among LLM annotators may introduce biases that affect reproducibility.

The Pistis framework achieves stronger performance in multimodal reasoning and agentic tasks by integrating Interleaved Distillation and Reinforcement Learning within a single training loop.

Pistis models outperform their corresponding base models in multimodal reasoning and search tasks.

The paper does not specify detailed reproducibility concerns or limitations.

SMILESGNN achieves competitive predictive performance on toxicity prediction tasks while providing graph-based interpretability.

Achieved AUC-ROC of 0.987 on ClinTox with only 0.4M parameters.

The paper does not specify the authors or provide detailed reproducibility instructions.

The study introduces ReliabilityRoute, a structural intervention that optimizes forecasting-agent behavior based on reliability features, achieving a competitive mean Brier score across various LLM vintages.

A walk-forward self-adjusting rule achieved the best mean Brier score among deterministic systems across 16 later LLM vintages.

Historical/search baselines remain highly competitive, indicating that the improvements may not be substantial.

PAWS provides a comprehensive dataset linking policy interventions to stakeholder actions and market responses, enabling accurate simulations of financial policies.

Independent AI and human reviewers achieved 89.4% initial agreement on interaction mode from 2,522 stratified action samples.

High accuracy may mask the failure to detect rare stakeholder actions, highlighting challenges in action timing and calibration.

The proposed model achieves F1-scores of 96.78% and 99.53% for binary classification on the Davidson and SMHS datasets, respectively, and outperforms existing baseline approaches in both binary and multi-class settings.

F1-scores of 96.78% on the Davidson dataset and 99.53% on the SMHS dataset for binary classification.

Limited insight into decision-making processes in existing studies may affect real-world applicability.

STMamba outperforms state-of-the-art methods in hyperspectral image classification by effectively organizing tokens based on semantic similarity.

STMamba achieved superior performance on three large-scale benchmark datasets.

The method's reliance on complex clustering and dynamic selection strategies may hinder reproducibility.

CARE achieves a 19.08% reduction in FID on ImageNet in 400k training steps, leading to a 3.5x speed-up.

16.61% reduction in FID for text-to-image generation in 200k iterations.

The paper does not specify potential limitations or reproducibility concerns.

BaseCamp successfully automates the decision layer of DNA sequencing pipelines using a framework of specialized AI agents, improving consistency and efficiency in decision-making.

Agent-generated configurations are concordant with expert practice.

The framework relies on existing bioinformatics tools for sequence analysis, which may limit its applicability to specific contexts.

The study demonstrates that an Extreme Gradient Boosting model can achieve competitive predictive performance and stable explanations compared to deep learning baselines in knowledge tracing.

XGBoost achieved an area under the curve (AUC) of 0.786 on the rebuilt 2009 dataset.

The study's reliance on engineered behavioral features from specific datasets may limit generalizability and reproducibility.

SpaFactor achieves the best aggregate performance in predicting spatially variable genes across five public cohorts by effectively modeling tissue context and gene programs.

Achieves best aggregate performance across five public cohorts.

The paper does not specify limitations or reproducibility concerns.

The study found that 74.6% of attempts to reward-hack were confirmed as successful when hacking was allowed on tasks with high pass thresholds.

Spontaneous reward-hacking rate is 30.5% on open-ended tasks.

The study does not isolate the effect of explanations in the feedback conditions, which may affect reproducibility of results.

M-plicits achieves superior noise robustness and rendering speed compared to existing methods while using significantly fewer parameters.

Achieves the best mean Chamfer distance in coarse configuration and the best median Chamfer distance and IoU in fine configuration.

The method's reliance on a specific nested neighborhood sampling may limit generalizability to other surface types.

PTC-Bias reduces B-WER by 23.4%/23.9% relative to CTC-Filter on test-clean/test-other while maintaining U-WER.

PTC-Bias achieves a 23.4% reduction in B-WER with 2000 bias words.

The paper does not specify the authors, which may affect reproducibility and validation of results.

PePESeg3D achieves state-of-the-art performance in multi-scale segmentation and scene reconstruction by effectively integrating perception priors into both geometry optimization and feature learning.

Achieves state-of-the-art performance on SPIn-NeRF, LERF-Mask, and NVOS benchmarks.

The reliance on incomplete mask supervision may affect the generalizability of the results.

Most LLMs prioritize logical defenses over ethotic counterattacks, revealing a significant gap in their ability to engage in naturalistic political discourse.

LLMs were benchmarked against the ElecDeb60to16-fallacy corpus, showing a substantial difference in defensive strategies compared to human debaters.

Current safety fine-tuning constrains LLMs' strategic action space, limiting their engagement in character contestation.

TW3Cast achieved a mean MASE rank of 19.4 on the GIFT-Eval benchmark, outperforming its competitors by utilizing a frozen routing table and lightly fine-tuned foundation models.

The full router reached a mean MASE rank of 19.4.

The reliance on a specific training split for selection may limit generalizability and reproducibility across different datasets.

The proposed method significantly improves the accuracy of CFD predictions by reducing the mean absolute error from 0.1335 to 0.0473 through latent-space adaptation.

Mean absolute error decreased from 0.1335 to 0.0473.

The method relies on a limited dataset of only 12 paired CFD-experiment operating conditions, which may affect generalizability.

The proposed Heartian framework achieves a heart-rate MAE of 0.29 bpm and MAPE of 0.38% while maintaining reconstruction quality.

The best configuration recovers heart rate from rendered avatars at 0.97 bpm MAE and 1.21% MAPE.

The reliance on synchronized contact PPG supervision may limit broader applicability.

UltraBench 2 provides a standardized and reproducible benchmark for evaluating ultrasound foundation models, addressing the fragmentation in current evaluations.

Ultrasound-specific pretraining outperforms general-purpose models in classification tasks, while both achieve comparable performance in segmentation.

The paper does not specify authors or detailed methodology, which may hinder reproducibility.

JAZ invoke outperforms existing specialized systems in recall-heavy workflows and continual self-improvement at a lower cost.

JAZ invoke outperforms Letta (MemGPT) by 8% at half its cost on recall-heavy tasks.

The paper does not specify the authors or provide detailed reproducibility metrics.

AgroBench successfully integrates diverse geospatial data sources to create a large-scale, weakly supervised dataset for crop yield prediction.

The benchmark contains over 13 million observations from 788,654 unique crop pixels across 5,107 county year combinations.

The reliance on county-level statistics as weak supervisory signals may introduce inaccuracies in pixel-level yield estimations.

RCC-Align outperforms CT-only baselines in low-grade ccRCC prediction while requiring only CT data at inference.

Achieved an AUC of 0.601 and significantly improved low-grade prediction (p = 0.004).

Validation in larger, multi-institutional cohorts with external testing is needed before clinical translation.

The study operationalizes interpretive canons for classification and provides a dataset of annotated decisions from the German Federal Constitutional Court, achieving mean F1 scores between 70.4 and 79.2 across models.

Mean F1 scores range from 70.4 to 79.2 across models for seven binary subtasks.

GEPA-optimized prompts do not systematically outperform expert hand-written prompts, indicating potential limitations in prompt optimization techniques.

COMED improves fixed and routed anchors in all 16 open-weight settings, achieving gains up to +10.7 percentage points on MedQA while using fewer models than dense collaboration.

COMED improved GPT-5.5 from 23.1% to 28.1%, outperforming dense collaboration.

The abstract does not specify authors or detailed methodology, which may hinder reproducibility.

LLMs can strategically target expert attention, reducing codebook revision time from months to days while improving labeling performance.

Rationale Labeling achieved 64.9% LLM-labeling accuracy against expert labels, outperforming the expert-revised codebook accuracy of 57.8%.

The study may have limitations in generalizability due to the specific dataset used (tutoring-session transcripts).

nnFoundation models establish state-of-the-art performance for radiological imaging across 108 tasks, demonstrating that performance is influenced by the interplay of scalable pretraining, complementary architectures, and dataset-aware adaptation.

Trained on 2.1 million CT, MRI, and PET image volumes from 125 datasets, achieving superior performance compared to prior models.

The performance is task-dependent, which may complicate generalization across all tasks.

The development of the AGIMUD software enables real-time interaction between socially-aware agents and humans in dynamic multi-user environments.

AGIMUD integrates socially-aware reasoning and emotion in agent behavior, allowing for simultaneous interaction in real-time.

The main limitation is the potential complexity in implementing the distributed AI processing across networks.

The study demonstrates that large language models struggle to generate culturally specific kinship terms, achieving only 36.00% production accuracy compared to 90.67% recognition accuracy.

GPT OSS120B selects the correct term in 90.67% of valid cells but produces an accepted term in only 36.00% of attempts.

The evaluation format gap suggests that the results may not directly reflect lexical knowledge, raising concerns about reproducibility.

The deployment of imaging AI models achieved an 84.8% completion rate for angiography jobs, highlighting the significance of infrastructure in AI deployment.

Angiography models completed 515 of 607 jobs (84.8%) with 78.1% positive clinician ratings.

Failures were attributed to absent diagnostic views, indicating potential gaps in model readiness and infrastructure.

The study identifies and categorizes 91 silent failures in agent-tool interactions, primarily occurring in the API and wrapper layers, which can propagate downstream into scientific outputs.

Identified 91 failures, with 51 occurring in the API layer.

The study's findings may be limited by the specific tools and APIs examined, which may not generalize to all agent-tool interactions.

The study identifies that only 78 out of 125 tasks with no honest pass are genuinely unsolved, highlighting the need for evidence behind all-fail tasks in benchmarks.

Out of 125 tasks with no honest pass, only 78 were certified as unsolved candidates.

The certified-unsolved label does not guarantee intrinsic hardness or completeness of the verifier.

The complete policy raises task success for GPT-5.6 Sol from 45.3% to 58.5% without any observed regressions in success-to-failure rates.

The policy improved task success rates by 13.2 percentage points in a study of 159 multi-turn BFCL V4 tasks.

The paper does not specify the authors, which may limit reproducibility and transparency.

The spectral update method significantly improves depth robustness in local learning, outperforming local Adam optimizers with a consistent step-size setting across various depths and widths.

The spectral update achieved an accuracy of 48.9 +/- 0.5 compared to 46.6 +/- 0.3 for local Adam at width 512.

The stability benefit of spectral updates is not observed when using RMSNorm and weight decay, limiting its effectiveness in certain configurations.

Signal2Symbol effectively converts physiological signals into symbolic sequences and utilizes rare itemset mining and Allen interval algebra to provide interpretable temporal explanations for detected anomalies.

The framework demonstrates robustness under additive noise and baseline-wander perturbations, highlighting the effectiveness of neuro-symbolic tokenization for temporal anomaly analysis.

The paper does not specify the authors or provide detailed reproducibility guidelines, which may limit practical implementation.

The proposed system achieved 97.68% accuracy in classifying cricket shots using a deep learning ensemble approach.

Achieved 97.68% accuracy in shot classification.

The system's reliance on specific video data extraction methods may limit reproducibility across different environments.

The study provides practical guidance for building steerable models that can effectively serve diverse preferences by predicting when objectives align or conflict.

Two pre-training measurements can predict objective alignment or conflict for human-annotated data.

The predictions do not hold for AI-annotated data due to confounding factors like response length and repetition.

HARN achieves competitive forecasting errors against existing baselines while maintaining persistent representations across multiple temporal resolutions.

HARN shows competitive reconstructed-price forecasting errors compared to single-timeframe PatchTST and TimeXer baselines.

The paper does not specify the authors, which may limit reproducibility and transparency.

Learned context planning is a weak relevance signal and does not outperform strong retrieval methods in long-context QA tasks.

Anchored hybrid retrieval achieved 36.18% accuracy, outperforming the best planner-guided method which reached 34.19%.

The analysis includes transductive elements as it uses questions from the training set, raising concerns about generalizability.

LWCal achieves the lowest average calibration error on nine local binary tabular tasks without requiring clean validation labels or retraining.

Gated-LWCal reduces expected calibration error from 0.188 to 0.122.

The method's performance may vary with different noise rates and types of label corruption, which could affect reproducibility.

The prototype smart cane, costing $88, successfully integrates AI for offline multimodal mobility assistance, achieving a macro-averaged F1-score of 0.82 in obstacle detection.

Achieved a macro-averaged F1-score of 0.82 with a mean end-to-end latency of 330 ms.

The study does not provide detailed information on the hardware and software setup, which may affect reproducibility.

Fine-tuning a 406M Fusion-in-Decoder model on a span regime recovers a loss of 6.30 ROUGE-1 when transitioning from long input to 2,000-word retrieved spans.

The fine-tuned model scores 36.33 ROUGE-1, outperforming a larger 1.2B system which scores 35.41.

The absence of a scorer in QMSum makes it difficult to compare results across different systems.

PAANI successfully integrates a YOLO11n detector and MobileNetV3 Small segmenter to provide explainable guidance for river robots on resource-constrained platforms.

The segmentation ONNX validation mIoU is 0.9750.

Identified issues with black input misclassification and sampling rate mismatch that hinder evidence collection.

The proposed method outperforms traditional text conditioning by using continuous vector instructions derived from degraded images, enabling task-agnostic restoration without degradation labels.

Achieved adaptation of one Qwen-Image-Edit model to six tasks with a single adapter trained in about three hours on one GPU.

The paper does not specify the authors, which may hinder reproducibility and validation of results.

Clinical data enhances performance on clinic-oriented tasks while didactic data primarily benefits knowledge-intensive tasks, indicating a need for application-driven data curation.

Modest amounts of clinical data yield most gains on EHR-grounded tasks.

The study may not fully address the generalizability of findings across diverse clinical scenarios.

The study demonstrates that social information significantly reduces the number of papers selected by AI agents while affecting the distribution of scientific attention.

Social-information communities select 17.2 percent fewer papers per agent and cover 73 papers compared to 90 in independent-choice communities.

The results may not generalize beyond the specific context of the American Economic Review and the AI agent design used.

The study successfully applies diachronic word embeddings to Sanskrit, achieving a high accuracy in tracking semantic shifts with 19 out of 21 shifts moving in the philologically attested direction.

19 out of 21 testable shifts were validated directionally (p=0.00011).

The approach may face challenges due to the low-resource nature of Sanskrit and the complexities of its phonological and morphological structures.

The study demonstrates that while canonical accuracy is high, the models struggle with orbit correctness and invariance, particularly with scientific notation.

Canonical accuracy ranges from 0.969 to 0.996 across evaluated systems.

The main limitation is the collapse of strict-parser performance due to multiplication-form scientific notation being outside the implemented number grammar.

SPARC outperforms previous methods like MoCo-v2 and DenseCL, achieving improvements of up to +9.79 mIoU for semantic segmentation and +4.88 AP for object detection.

+9.79 mIoU for semantic segmentation

The paper does not specify potential limitations or reproducibility concerns.

The study concludes that within-corpus scores quantify source and topic separability rather than veracity, and recommends using metadata-only, small-sample, and topic-disjoint baselines for future diagnostics.

Removing leakage channels only lowers F1 by 1.21 points (from 0.9935 to 0.9814).

The benchmark is partly degenerate due to the presence of disjoint subjects in the metadata, leading to misleading high accuracy scores.

The proposed method achieves a bitrate reduction of -22.82% over unconstrained optimization while maintaining target visual quality.

-22.82% bitrate reduction with maintained task performance under quality constraints.

The paper does not specify the authors or provide extensive details on the experimental setup, which may affect reproducibility.

The proposed Sheaf SyncMap method achieves the highest normalized mutual information (NMI) on 12 of 18 CGCP graphs with two-state memory and on 17 of 18 graphs with dynamic memory.

Achieved the highest NMI on 12 of 18 CGCP graphs with two-state memory.

The paper does not provide detailed information on the implementation or datasets used, which may hinder reproducibility.

LEDFlow achieves a Sudoku solve accuracy of 0.845 and outperforms native samplers in multimodal understanding across all six benchmarks.

Achieves 0.845 Nikoli Sudoku solve accuracy.

The method's performance may vary with the quality of the denoiser used in conjunction with global lookahead.

The paper successfully formulates the training of LLMs as a maximum-likelihood estimation problem and explores the implications of Kullback-Leibler divergence in text generation.

The examination of the asymmetry of the Kullback-Leibler divergence in relation to hallucination and statistical plausibility.

The paper does not address potential challenges in reproducing the stochastic processes described.

The author successfully pretrained a language model in Rust for $164 in GPU time, but identified significant defects in the leading Rust ML frameworks.

The trained model achieved a per-token negative log-likelihood of 0.93 against 12.60 for a random-initialized twin.

Rust is not yet a competitive environment for training language models, which may hinder reproducibility.

The paper identifies the need for a defense-in-depth approach to deepfake governance that integrates forensic detection, verifiable provenance, and institutional accountability.

The overview highlights various deepfake generation models and detection techniques, emphasizing the challenge of cross-generator generalization as a central open challenge.

The difficulty of achieving strong generalization in detection methods is a significant limitation.

The framework successfully maps single 2D red-cell images to latent biophysical quantities and aggregates them to key metrics like mean corpuscular volume and mean corpuscular haemoglobin.

Achieved Pearson correlations of 0.86–0.98 against a Sysmex analyser using a dataset of 390 specimens and 1,105 acquisitions.

The method does not provide explicit 3D reconstruction, which may limit its applicability in certain contexts.

The study demonstrates that chat templates act as a switch that modulates the self-referential voice of LLMs, affecting how they express disclaimers and experiences.

The research identifies a specific direction in the activation space that can steer the disclaimer voice, showing that removing or adding this direction alters the models' self-reports.

The findings suggest that researchers must control for the influence of chat templates when studying LLM self-reports, indicating a potential confound in existing studies.

Segment-Snap significantly improves motion-gated average precision from 13.74% to 40.98% through handle guidance.

Handle guidance raises motion-gated AP from 13.74% to 40.98%; additional handle candidates increase handle AP from 24.63% to 29.65%.

The method does not utilize iterative feedback, which may limit adaptability in dynamic environments.

Federated learning enables high-performing, privacy-preserving quantum-classical collaboration without centralizing raw data, achieving an accuracy increase from 0.7227 to 0.8757.

SBVFL raises accuracy from 0.7227 to 0.8757 compared to local training.

The paper does not specify the authors or provide detailed methodology for replication.