Within the Creator Highlight collection, TDS Editors chat with members of our group about their profession path in knowledge science and AI, their writing, and their sources of inspiration. As we speak, we’re thrilled to share our dialog with Shuai Guo.
Shuai is an industrial AI researcher working with physics, knowledge, and machine studying to unravel real-world issues in engineering, safety, and clever methods. He holds a PhD on the intersection of computational mechanics and machine studying. His work spans varied matters, together with anomaly detection, digital twins, physics-informed studying, and LLM/agentic purposes.
Your LangGraph piece walks the reader via the method of constructing a deep analysis agent. Whenever you really tried it end-to-end, what shocked you probably the most, and what would you do in a different way subsequent time?
I might say what shocked me probably the most was how simply the deep analysis agent could make errors when operating it end-to-end. That entire “generate question → search → replicate → repeat” loop appears nice on paper, but it surely falls aside fairly quick. There are two fundamental points I bear in mind clearly. First, infrequently, the agent begins mixing up what it discovered with what it remembers from pre-training. This isn’t supreme, as I solely need the LLMs to synthesize data and determine information gaps, whereas absolutely relying on the net search to floor the reply.
One other situation that consistently offers me complications is data contamination, i.e. when search brings again related stuff however the mannequin treats it prefer it’s precisely what you requested for. For instance, I as soon as examined the deep analysis agent by researching a particular bug report (say, situation #4521 of a codebase), and the search would return content material associated to situation #4522 and begin mixing of their signs like they’re all the identical drawback.
Past these two fundamental points, I additionally skilled challenges in dealing with conflicting data and figuring out sufficiency for terminating the deep analysis. None of these issues will be solved by merely including extra search outcomes or operating extra iterations.
The important thing realization for me is that guardrails are as crucial, if no more so, than the agent structure, if we need to transcend “only a demo” and construct a system that truly works in manufacturing. I feel the mindset of “test-driven growth” matches properly right here: outline what “good” appears like earlier than you construct. Subsequent time, I’d begin by defining clear guidelines, after which construct the agent structure round these constraints.
You’ve written that analytical AI (SQL/BI + classical ML) isn’t going away just because agents are hot. Should you had been designing a contemporary knowledge stack right this moment, what work would you give to brokers and what would you retain within the analytics lane?
Analytical AI is reproducible and numerically exact. LLM-based brokers, alternatively, are good at digesting unstructured context, translating outcomes, and speaking with individuals. For allocating duties between analytical AI and agentic AI, I might say if a process is extra quantitatively geared, I might default to analytical AI; but when it’s extra qualitatively geared, e.g., synthesis, storytelling, or judgment, I might take into account LLM/brokers as higher options.
We will take into account a concrete drawback of constructing a buyer churn prediction system. On a excessive degree, it often includes two steps: figuring out the at-risk prospects, and performing on them. For step one of flagging at-risk prospects, I might lean on analytical AI to engineer informative options, prepare gradient boosting fashions on historic behavioral knowledge, and use the skilled fashions to calculate churn propensity scores. As well as, I might additionally run a SHAP evaluation to get function significance scores for explaining the prediction. Each step is exact and reproducible, and there are a ton of finest practices out there for getting correct and dependable outcomes.
However then comes the enjoyable half: what do you really do with these predictions? That is the place the LLM-based brokers can take over. They’ll draft customized retention emails by pulling within the buyer’s historical past, possibly counsel related product options they haven’t tried but, and regulate the tone primarily based on how their previous assist tickets went. There isn’t a math right here. Simply talking in a contextually good approach.
What’s one ability you invested in early that now offers you a bonus as AI instruments get extra succesful?
Methods considering.
To me, methods considering is mainly asking methods to decompose methods into elements. How do completely different elements speak to one another? What are the handoff factors? The place are the suggestions loops? If I contact this, what else adjustments?
I picked this up at college. I majored in aerospace engineering with a deal with aero-engine design. The factor about jet engines is that every thing impacts every thing, and learning it actually helped me develop three habits: decompose the system, outline clear interfaces, and all the time look out for coupling results.
It’s true that AI instruments are getting extra succesful, e.g. we bought higher coding assistants, more practical RAG pipelines, or LLMs that may deal with longer context, however many of the developments occur in slender slices. As a substitute of all the time chasing the most well liked software and attempting to include it someway in my current work, methods considering helps me put the massive image entrance and middle. For an LLM utility, I might all the time begin by sketching the elements, figuring out interplay and inputs/outputs between the elements, ensuring checks and guardrails are added, after which swapping elements as instruments enhance.
The truth is, constructing LLM purposes jogs my memory numerous designing jet engines: new expertise comes and goes, however a stable system design compounds worth.
Should you zoom out, what a part of knowledge science or AI is altering too quick proper now, and what half isn’t altering quick sufficient?
I feel multi-agent AI methods are positively one of many hottest fields which might be shifting very quick. We see fancy demos (be it coding assistant or analysis assistant) every so often. New open-sourced frameworks that allow builders to effectively construct their very own multi-agent purposes additionally pop up consistently. All of that is thrilling. However right here is the factor: are we pushing out these difficult methods approach sooner than we perceive how they’ll really behave in observe?
That’s the place I see the hole: the entire “assurance” layer round these multi-agent methods isn’t evolving quick sufficient. To deal with this problem, we will (and possibly ought to) deal with these multi-agent methods identical to every other industrial system. Within the manufacturing trade, it’s a widespread observe to undertake data-driven approaches to help system design, management, situation monitoring, and fault evaluation. This similar strategy may benefit multi-agent methods as properly. As an illustration, how about we use Bayesian optimization to design the multi-agent structure? How about utilizing ML-based anomaly detection to observe the brokers’ efficiency and catch safety threats?
The excellent news is there’s momentum constructing. We’re seeing observability platforms for LLMs, analysis frameworks, and so forth., and so they’re laying the groundwork for making use of these industrial-grade, data-driven strategies. I see numerous alternatives on this house and that’s what will get me excited: the possibility to convey the rigor of business methods to agentic AI and make these instruments dependable and reliable.
To be taught extra about Shuai‘s work and keep up-to-date along with his newest articles, you possibly can comply with him on TDS or LinkedIn.
