What, Why and ahead...

3rd Year, Conundrum & Neural Ordinary Differential Equations

David Bowie (for those too tired to click the links)

David Bowie disappeared from music (to indulge in other creative avenues) for nearly a decade before returning with The Next Day, everyone thought it was the end but he had

Blackstar scheduled for his 69th birthday. Who would have thought that would become the permanent end of his legacy?

I admire him far too much to initiate that release schedule, so here's a post after only 1 year of disappearance, not ten.

Somewhere between entering in second year, juggling different technical fields and trying to understand what exactly I'm doing with my life, writing quietly dissolved into the background (or did it ?).

But unlike vanishing gradients, this disappearance never fully converged; somewhere underneath all the noise, the loss was still propagating.

Bowie might not have had the opportunity to properly bid farewell to his listeners, but I apparently have all the time in the world to justify my disappearance with a Bowie metaphor, so here's how the tale goes.

As the youthful delinquency of the first year of college slowly faded into the summer of 25' , following a quick trip to the state capital with one of his mates, the writer found himself standing at the threshold of the most cliché engineering question imaginable:

"What now?"

With his twisted tendency to obsess over the smallest details and inflate them into matters of existential importance, he embarked on a pursuit to find something worthwhile.

In the beginning, the criterion was stupid simple:

"If one possesses a functioning laptop and an equally overactive mind, one might as well explore the things lying at the intersection of the two."

Now, the writer would very much like to frame this as some methodical tautology rather than a glorified monkey-see-monkey-do heuristic, but we both know what was actually going on here.

So, like any oblivious sophomore engineering student, the writer descended into the convoluted jungle known as the Internet, actively searching for ways to put his resources to work. 

Now, the writer would once again love to expand on his journey like Odysseus returning to Ithaca, but that would be painfully pretentious (and he likes it, a twisted one indeed).

Anyways, aside from the Odysseus-like adventure, he did find many avenues for himself. What he discovered, however, was painfully simple. The dense jungle of possibilities quickly cleared itself into a single glowing neon sign, casually stating:

"Congratulations on your mechanical engineering degree!!

But have you heard that Clipkart offers 16 LPA base for fresher SDE roles?" 

(The writer may turn into a rogue philosopher after this; reader discretion is advised.)

The recurring pattern eventually became difficult to ignore.

Every road that managed to hold the writer's attention for longer than a few weeks seemed to share strange characteristics. This involved mathematics that looked intimidating enough to scare away sensible people (trying to make a career in it without being called a NERD), statistics that carried hand-wavy explanations for divisions by reduced quantities, graphs that transformed numbers to stories and, quite often, programming languages obscure enough to trigger the question:

"Who exactly is using this thing ?"

The writer found himself being naturally driven to the centres of internet where people discussed collected numbers with and almost unsettling level of enthusiasm. They used mathematics through computers to make predictions, uncover hidden patterns, model real-world phenomena and occasionally convince themselves that reality could be compressed into a sufficiently elegant distribution.

What fascinated the writer was not merely the subject matter, but the people. Unlike endless parade of tutorials, interview guides and placement-oriented roadmaps, these communities seemed genuinely interested in understanding something. 

The connection between between their work and real world felt natural and driven entirely by curiosity of the utmost kind; they were actually bridging the gap between formal learning in class with applications in a way that felt organic, almost as if the applications themselves were extensions of the theories being developed.

Learning was not treated as an obstacle to overcome before doing meaningful work. It was an essential part of the process.

What followed was roughly a year of intellectual pinball. 

The writer bounced between disciplines with the grace of a particle undergoing brownian motion. Control theory, machine learning, statistics, applied mathematics, optimisation, scientific computing, a brief flirtation with music production and enough abandoned side quests and half-baked projects to populate a moderately successful LinkedIn profile.

Most of these pursuits shared a common lifecycle.

"A fascinating idea would appear. The writer would spend several weeks convinced he had discovered his calling. This conviction would survive precisely until the moment genuine complexity emerged from behind the introductory material. At which point the fascination would either mature into sustained interest or collapse under its own hype." (one can check the progress here.)

Just like any other thing in life, if one fucks around with many things, a few paths are bound to lead somewhere.

The survivors all revolved around the same fundamental premise: constructing mathematical descriptions of reality and then convincing computers to do something useful with them.

With this, the writer proceeded into his second year of engineering, carrying a fresh arsenal of interest, a newer direction and an unreasonable level of optimism.

Attempting to document the entirety of that period would be an exercise in historical revisionism. Among the notable developments were the institute's enthusiastic adoption of zero-attendance policy, a steadily declining academic trajectory, a questionable relationship with stimulants and, ironically, a persistent sense that everyone else had received an instruction manual for life which the writer had somehow misplaced. 

Fortunately for both the reader and the writer, those events belong to a different story.

This one concerns the mathematical descriptions of reality that survived the chaos.

At the very end of his remarkably unstructured journey, the writer was presented with his first genuine opportunity to engage with technical work driven almost entirely by curiosity rather than placements or the latest industry trends: assisting a professor with a research project.

Surprisingly, the countless hours spent wandering on the obscure corners of the internet had accumulated into something resembling competence. The assorted projects, notes and intellectual detours that once appeared completely disconnected suddenly formed a portfolio convincing enough to secure a position on the project.

Even though he was conscious about choosing the right professor (following careful consideration of his interests and a sophisticated market analysis of his batchmates), the problem that finally landed on his desk appeared deceptively straightforward at first. 

"It was not."

What initially seemed like a simple controller design unravelled slowly into an uncomfortable collision between differential equations, dynamical systems, numerical methods, a completely new programming language and enough unfamiliar notations to make the writer question whether he had accidentally opened the wrong paper.

Enter the third part of the title, "Neural Ordinary Differential Equations" (will try my best not to repel anyone.)

What could have been another summer vacation spent in mindless exploration? has suddenly been given a direction.

For the first time, the writer didn't have to look for his next muse; rather, he was handed a problem which didn't care if he was prepared.

Seemingly disconnected fragments from entirely different disciplines collided to produce something neither field appeared willing to claim responsibility for. Differential equations met machine learning, and control theory shook hands with numerical methods, and somewhere in the resulting chaos emerged a concept the writer had never encountered before.

Neural Ordinary Differential Equations.

NOTE: This is a demonstration of the final product of last month's work; a neural network, not a conventional controller, is fully responsible for moving the beam.