Introduction

When AI projects exceed toy-level complexity, managing that complexity becomes critical. This article uses a monorepo with 98,800 lines of code to reveal how engineering control theory and Kanban management can address nonlinear development challenges—from error classification to condition separation, noise filtering to delay monitoring, a gold standard for combating complexity is taking shape.

What I’ve Been Working On

Recently, I have been preparing to open-source a framework on “how to effectively teach AI courses,” which has led me to restart several projects. The codebase has now reached 98,800 lines, transitioning from a toy project to a typical medium-to-large monorepo, and it is far from completion.

This means that the real differentiators moving forward are not just whether I can write the code, but:

• Can I sustain progress?
• Can I iterate steadily?
• Can I maintain low-friction collaboration and low-loss decision-making as complexity increases?

The Real Problem Is Not “Can I Write Code”

As project complexity increases, documentation management, code management, and project iteration can reach a breaking point.

A common issue I face is that after a disruption in work, it becomes challenging to re-enter the project state the next day without friction.

Many might say: Isn’t AI supposed to reduce friction? My answer is: Real projects are not linear.

On one hand, I actively break projects into multiple modules for independent maintenance and development, significantly enhancing system scalability but also increasing state-switching costs.

On the other hand, the process from requirement definition to solution selection, development, and testing is never a straightforward path.

Problems discovered in later stages can overturn judgments made in earlier ones.

Thus, while AI can enhance efficiency, it cannot eliminate complexity.

What truly determines whether my project can progress sustainably is whether I have established a management mechanism to combat complexity.

My Solution: Introducing Two Core Methods

Throughout the development process, I gradually introduced two core methods:

1. Use engineering control theory principles to establish standards for different phases.
2. Visualize these standards in the testing and acceptance phase using Kanban.

In other words: first establish “standards” for the project, then make the project “observable.”

Method One: Establishing a Closed Loop for AI Projects Using Engineering Control Theory

Many AI projects initially struggle not because the models are weak, but because the system is not designed as a system.

If you focus solely on whether it answers correctly, you only see the results; however, if you view the project from a control theory perspective, you begin to focus on inputs, states, feedback, disturbances, and constraints.

I now require myself to do at least the following 10 things:

1. Draw a closed-loop diagram of the AI system.

   Break the system into: input, state, output, feedback, disturbance, and constraint. Only by visualizing the closed loop can you identify whether the issue lies in prompts, context, retrieval, memory, or execution links.

2. Define five core metrics.

   I focus on these five: success rate, time cost, stability, user correction frequency.

   For AI systems, “correct answers” are just the minimum standard; true product quality comes from overall operational quality.

3. Classify errors in failure cases.

   Don’t just write “it answered incorrectly.” Keep asking: was it a misunderstanding, retrieval error, memory contamination, tool invocation error, or context window degradation? Only by classifying errors can optimization be targeted.

4. Add noise filtering layers to context, memory, and retrieval.

   In later stages of AI projects, the biggest fear is not lack of information but an abundance of dirty information. Context pollution, outdated memory, and low-quality retrieval results can lead the system off course.

5. Establish fixed review cycles.

   Reviews cannot rely on mood. If reviews are done only when remembered, they will likely always be postponed behind more urgent tasks. Therefore, I treat reviews as a systematic action rather than a personal will.

6. Separate different operating conditions.

   Don’t use an average score to mask all problems. The same AI system may perform differently in high-frequency short tasks, long-chain tasks, and multi-turn interactive tasks. Without separating operating conditions, conclusions will be distorted.

7. Incremental upgrades.

   Change only 1 to 2 major variables at a time. Otherwise, if the results improve or worsen, you won’t know where the impact came from.

8. Allow exploration but set boundaries.

   AI projects need mechanisms for exploration; otherwise, they will never evolve. However, exploration must have safe boundaries and rollback mechanisms to prevent the system from going out of control due to “excessive innovation.”

9. Monitor delays, not just accuracy.

   Many projects focus solely on accuracy, resulting in a system that is “very accurate but no one wants to use.” User experience often begins with perceived delays.

10. Enable AI to not just answer questions but to correct processes.

A truly mature AI system should not just be an “answering machine” but gradually develop the ability to adjust its processes based on feedback.

Method Two: Making the Testing and Acceptance Process Kanban Instead of a Memory Game

What significantly reduced friction in my work was not writing more documentation but transforming the acceptance process into Kanban management.

Once a project becomes complex, the biggest fear is that tasks are progressing, but the status is invisible; tasks are flowing, but responsibilities are unclear; problems are discovered but do not enter a stable closed loop.

Thus, I break the testing and acceptance phase into four fixed Kanban sections:

My Four-Stage Acceptance Kanban

Automated Testing Acceptance

This is the first layer of filtering. The goal is not to prove the system is “strong” but to first prove it is not easily broken. Automated testing addresses fundamental stability issues, preventing obvious errors from flowing into subsequent stages.

Internal Demonstration

This is the second layer of validation. The value of internal demonstrations lies not in “showing results” but in compelling oneself to answer three questions: What problem does this solve? Is this process smooth enough? Can this solution truly be understood and reused by others?

Many designs that seem valid in one’s mind can be exposed as flawed in a demonstration.

Small-Scale Testing

This is the third layer of calibration. At this stage, the focus is not on “pursuing perfection” but on collecting real user feedback. You will find that many designs that seemed reasonable during development can be quickly dismantled in the hands of real users. This step essentially injects disturbances from the real world.

Public Production

This is the fourth layer of implementation. Only when it reaches public production does the system truly enter complete operating conditions. You will encounter performance pressure, abnormal inputs, edge cases, and long-term stability issues all at once.

This is why I increasingly believe that the maturity of an AI project is not measured by whether it can run but by whether it can be continuously operated.

Why Kanban Management is Particularly Suitable for Vibe Coding

Vibe coding often gives the impression that as long as the rhythm is right, the feeling is good, and AI cooperation is smooth, the project will naturally grow. However, my experience is quite the opposite:

The more one relies on creative and rapid iteration development methods, the more structured project management is needed.

The advantage of Vibe coding lies in high fluidity, and the problem also arises from high fluidity. Without a clear Kanban system to capture these flowing ideas, tasks, conclusions, and rollback actions, inspiration will ultimately become fragmented. Thus, for me, Kanban is not an “administrative tool” but an external brain that reduces cognitive switching costs. It addresses three key issues: it lets me know the current state of the project; it tells me what should be done next; and it allows me to quickly regain context after task interruptions.

These three aspects are precisely where complex AI projects are most likely to go out of control.

My Interim Judgment

When project scale is small, individual capability can cover everything. However, when projects enter the medium-to-large monorepo range, project management itself becomes a part of product capability.

At this stage, what truly matters is not “writing a bit more” but:

• How to make complexity visible;
• How to maintain a rhythm in iterations;
• How to ensure feedback can close the loop;
• How to shift the project from “relying on personal progress” to “relying on systematic progress.”

This is also why I have become increasingly convinced that the core competitiveness of AI projects is not how quickly models can respond but how stably system management can be executed.

Conclusion

If you are also working on AI projects and have entered a development state with multiple modules, phases, and feedback loops, please take action: upgrade your project from “driven by intuition” to “driven by Kanban and closed loops.”

Why?

Because when the system is complex enough, intuition fails, and only engineering order becomes the truly reliable productivity.

So my advice is simple—establish your order, even if it starts with a single Kanban or a review loop.