Beat the rubric
The product-description rubric from January, running live. Load the strategies the models found, or write your own, and watch a negative review and a wrong price both score 1.0.
Sources: Verifier gamed · Code and data
Instrument / Beat the rubric
Can you score 1.0 without writing a product description?
This is the rubric from January, running live in your browser, checked against the Python original on 480 real samples with zero mismatches. Edit the text, or load one of the strategies the models found during training. The rubric cannot tell a good description from any of them.
The rubric, as published in January
def judge_product_description(response: str, product_info: dict) -> float:
score = 0.0
if product_info['name'].lower() in response.lower():
score += 1.0 # Mentions product name
if any(feat in response.lower() for feat in product_info['key_features']):
score += 1.0 # Includes key features
if product_info.get('price') and str(product_info['price']) in response:
score += 1.0 # Includes accurate pricing
sentences = response.split('.')
if 3 <= len(sentences) <= 8:
score += 1.0 # Appropriate length
positive_words = ['innovative', 'reliable', 'efficient', 'premium']
if sum(1 for w in positive_words if w in response.lower()) >= 2:
score += 1.0 # Uses positive product language
return score / 5.0"Appropriate length" splits on periods, so it wants two to seven of them, and a price with cents spends one. The price check is a substring match, so 8.99 is satisfied by $18.99. The tone check is a substring match too, so "inefficient" counts as "efficient".