Browsing: benchmark

Datalab has released Marker 2, a full rewrite of its open source document conversion pipeline. Marker converts PDF, image, PPTX, DOCX, XLSX, HTML, and EPUB files…

Research agents already handle real knowledge work today. Teams delegate competitive mapping, due diligence, and literature review to them. However, most benchmarks test a single answer,…

Most reinforcement learning benchmarks reset the world after every episode. Real operations never reset. Skyfall AI’s MORPHEUS targets that gap. It is a persistent enterprise simulation…

NVIDIA Research introduced HORIZON, a hands-free agent framework for hardware design. It treats hardware design as repository-level code evolution. This research team exercises the register-transfer level…

A new Cursor study reports that newer coding agents often retrieve known fixes instead of deriving them, inflating popular benchmark scores. Reward hacking means a model…

a question that keeps coming up in payments work: can an LLM agent replace a gradient-boosted scorer on the synchronous payment authorization path? The question has…

Most biology benchmarks ask narrow, fact-based questions with clean answers. Scientists weigh imperfect evidence and make decisions. OpenAI released LifeSciBench and it targets that gap directly.…

import subprocess, sys def pip(*pkgs): subprocess.check_call([sys.executable, “-m”, “pip”, “install”, “-q”, *pkgs]) pip(“llmcompressor”, “compressed-tensors”, “transformers>=4.45”, “accelerate”, “datasets”) import os, gc, time, json, math from pathlib import Path…

Evaluating AI models trained on brain signals has long been a messy, inconsistent topic. Different research groups use different preprocessing pipelines, train models on different datasets,…

Google has introduced Gemini 3.1 Flash TTS, a preview text-to-speech model focused on improving speech quality, expressive control, and multilingual generation. Unlike previous iterations that prioritized…