Qodo Flow Technical Q&A

Prompt vs. Flow Engineering

Q: How much time was spent on "prompt engineering" compared to "flow engineering"?

A: Approximately 95% of the effort was dedicated to high-level design and flow engineering.

Data Leakage Prevention

Q: How is data leakage avoided?

A: The test set includes problems from after September 2021, aligning with GPT-4's data cutoff, ensuring no leakage specifically for GPT-4.

Language Agnosticism

Q: Is AlphaCodium limited to specific programming languages?

A: No, the flow is language agnostic and can be applied to any language.

Context Window Management

Q: How is the context window managed?

A: Models with an 8192 token context window were used. Larger contexts may lead to some information being ignored.

Iteration on Code vs. Tests

Q: Why iterate only on generated code and not on the AI-generated tests?

A: For code problems in CodeContests, iterating on code is more beneficial than on tests, which are simple input-output pairs.