problems
- experiment tracking (weights & biases)
- what it is: logs training metrics, checkpoints, and run history so you can compare experiments and reproduce a result later. without it, a run is unrecoverable, you can't tell which config made which outcome.
- cost + what you get: enterprise per-seat pricing, six figures a year at scale. the api covers metric logging, checkpoint and artifact storage, run comparison dashboards, hyperparameter sweeps, report sharing.
- the catch: most teams only use a slice of that, mainly metrics plus artifact storage, so the price buys a lot of surface nobody touches. going open source doesn't just fix it though. self-hosted trackers are unproven at high write-frequency logging (sub-5-second intervals), and the real cost driver isn't the metrics, it's heavy media, camera and video streams. host that yourself and the storage bill can land close to what you're already paying.
- need: a tracker that's cheap at the metrics layer and actually built to carry heavy media at high logging frequency, not bolted on after the fact.