Research Principles
There is no "success" or "failure" in research, only insights and confidence levels.
Hypothesis-Driven Exploration
- •State hypotheses explicitly before running experiments
- •Pre-register predictions to avoid post-hoc rationalization
- •Document negative results - they're data too
Red-team Your Results
- •Define the scope: how sensitive is your finding to prompt variations, tasks, models?
- •Actively seek disconfirming evidence
- •A single observation is an anecdote, not a conclusion
Documentation Standards
- •Log everything: commands run, parameters used, timestamps
- •Verbatim outputs over paraphrasing
- •Separate observations from interpretations
Avoid Common Pitfalls
- •Confirmation bias: Actively seek disconfirming evidence
- •Cherry-picking: Don't ignore results that don't fit
- •Over-interpreting: Single observations are anecdotes, not conclusions
- •Blind fixing: Don't try random fixes without understanding the root cause