Time flies, especially when you’re enjoying it. It’s already been two years since I decided to join Princeton. Now that I’m in the second half of my second year, I want to look back on how I’ve been “philosophizing” this Doctor of Philosophy journey.

My first year was full of pure joy and harsh rejections. Those two words feel diametrically incompatible at first glance, but my experience doesn’t make sense without both.

What I wanted from year one

I started my PhD with two goals: (1) publish a paper in my first year, and (2) fully experience American college life.

I set the first goal because I wanted to get my research going and integrate into the group as soon as possible. But it was much, much harder than I expected, which I’ll get into later. The second goal felt uniquely possible at Princeton, even as a PhD student.

In most European countries, doctoral students have formal contracts with the university. That means you’re technically an employee, and you’re treated like one. Most things feel transactional. Universities typically don’t actively organize social events or build community, so many PhD students mostly interact within their research groups.

Princeton feels very different. It builds a strong community around grad students. The graduate school is small, around 3,000 students in total (even smaller than Stanford’s med school). We’re all part of the Grad College, which organizes a steady stream of social and academic events. Most campus events and clubs are also open to both undergrads and grads. For example, I’ve been hiking and skiing with the outdoor club, which is founded by the university but primarily organized by undergrads.

Most importantly, virtually everything is free for students. Princeton is very rich, and it treats its students very well. Events like this would be hard to imagine at most European universities (including top-ranked ones).

Year one in numbers

• Number of paper submissions: 8 (4 as first author)
• Number of paper rejections: 6
• Number of grant proposals I helped with: 2
• Number of undergrads mentored: 4
• Number of books read: 9
• Number of courses taken: 6 (required)
• Number of high tables attended: 5
• Number of grad formals: 1
• Number of hikes: 17
• Number of skiing/skating trips: 2
• Number of concerts on campus: 2
• Number of free meals on campus: +∞

A typical day

• 8:00 AM — Play tennis with Minhao at the Princeton Racquet Center
• 10:00 AM — Take the Tiger shuttle to my office
• Morning — Work
• 12:00 PM — Attend various seminars or talks on campus
• 1:00 PM — Get free food :)
• Afternoon — Work
• 5:00 PM — Take the shuttle back to my apartment on the Meadows campus
• 6:00 PM — Hit the Meadows gym or play squash
• 8:00 PM — Cook dinner or get free food again :)
• Evening — Work

Rejections

Publishing a paper in my first year turned out to be much harder than I expected, especially because I was working in a research area that was still relatively new to me. I used to think computer networking was basically similar “systems,” just with different domain applications. Oh boy, I was wrong. The way people in this field approach problems and assign value is different (e.g., empirical performance optimization vs. formal guarantees), and it took me quite a while to adapt.

To this day, I feel extremely fortunate to be my advisor’s student. She is one of the smartest and hardest-working people I’ve ever met. More importantly, she’s a kind mentor. She stays patient when I get stuck, and she’s willing to learn together with me, especially because what I’m working on spans several fairly different subjects.

Early on, I was also a difficult student to work with. I tend to have strong opinions about research direction and experimental methodology. At the time, I didn’t fully understand the difference between the community I came from and the computer networking community. I didn’t know how to present my ideas as a research paper, instead of a technical report packed with engineering details.

I still remember my first submission. Right before the deadline, my advisor had to rewrite most of what I had written, because my writing was so off track. She walked me through the process and even stayed up with me until 4am. I sincerely appreciate her patience and how she kept pushing me to think differently. I also remember vividly the moment I got my first rejection. Two seconds later, I received a message from my her: “I’m sorry about the paper.” In that moment, something clicked. She wasn’t my boss, she was my mentor.

“Your ideas are like your children, and you don’t want them to go into the world in rags.” — Patrick Winston

Rejection is always hard, because our ideas are our children. But over time I realized the sting wasn’t just “they said no.” It was being forced to look at my own work with fewer illusions: What is the core claim, really? What is the clean story? What would a skeptical reviewer misunderstand, and why would that be my fault? Each rejection pushed me to tighten the logic, trim the noise, and crystallize the contribution in writing. In that sense, the reviews were less a verdict and more a mirror.

So when the paper was finally accepted, I felt genuinely happy, but not surprised. By then, the work had been stress-tested enough that acceptance felt like the outcome finally caught up with the version of the paper we had fought to make real.

I also learned not to over-interpret any single decision. There’s randomness in the review process, and with LLMs entering the workflow, that noise can feel even louder. A rejection doesn’t automatically mean the work is bad. It might mean the fit was off, the framing didn’t land, or the message got lost. The only part you can control is the slope: keep learning, keep revising, and keep making the work clearer and stronger.

Still enjoying my solitude

My life has become much more colorful since I joined Princeton. I’ve made a lot of new friends and picked up new hobbies. But something has stayed the same. Unlike what many people worry about (loneliness), I’m terrified of someone getting into my life too much.

I’m always alone but never lonely, as I enjoy solitude more than anything. I love jogging alone around Lake Carnegie and along the Delaware Canal towpath (with many 🦊s and 🦌s). I love walking from my apartment to my office in the morning, and walking back alone at night. I also love standing alone in front of Lake Carnegie and letting the sunshine touch my face.

Being alone gives me time and space to think and relax. It’s hard for many people around me to understand (my mom included), but to me, time alone is emblematic of unbridled freedom and control over my own life. In the end, life is ultimately a journey you take alone, whether you like it or not. If you haven’t found joy in solitude, I hope you find your own version of it, even in small doses.

PS: I also realized I’ve reached an age where the way I treat friends (especially female friends) can lead to misunderstandings. So I’ve decided to keep a necessary distance, to carve out space.

Finding meaning in a PhD (and in life)

To me, doing a PhD at a place like Princeton is an absolute privilege. It gives me time and space to think, and to reflect on myself: what I really want, what I’m good at, and what I’m not. For me, a PhD is not about innovation, “moving the needle,” or advancing humanity. It’s about understanding, understanding science, and maybe most importantly, understanding the science of life.

Since coming to Princeton, I’ve started reading physical books again. Thanks to a good friend (Patty 🙏), I discovered Gödel, Escher, Bach: an Eternal Golden Braid, and I got hooked. This book is about formal logic and its connections to math, art, and thinking. What caught my attention is that mathematical and logical symbols possess no inherent meaning. But once we give them meaning, they can express all of mathematics and beyond.

This got me thinking: why would life be any different? As humans, we have moved past the primal search for our next meal or the mere drive to find mates to reproduce. I believe life does not have an inherent, predetermined meaning unless we actively assign meaning to the world around us.

Life itself is difficult, it’s hard, it’s challenging. This treacherous journey comes with obstacles and all kinds of internal and external struggles that we have to endure. If we don’t give it meaning, then what is the point of being here and going through it all?

Conclusion and afterthoughts

My first year was full of pure joy and rejection. I loved it, and I enjoyed it wholeheartedly. I came to understand my research and myself much better. I’m grateful to Princeton for giving me a safe and enjoyable space to think with a peaceful mind. I’m grateful to my advisor and my friends for creating a supportive and intellectually stimulating environment.

As AI has been upending the world as we know it, an ivory tower like Princeton has shielded us from much of the chaos. It gives us space and time to look inward, rather than constantly worrying about where the world might be heading. We will all have to face that chaos eventually, but for now, I cherish this space.

When I am old and can no longer be as active, I hope to have a rich memory book full of beautiful moments to flip through, rather than just a seemingly dense CV. Life is not a race to a finish line, nor is it merely about grinding through the daily motions. I’ve come to realize it is neither about the process nor the destination; it is about understanding who we are, and who we will eventually become along the way.

Prelude

Two years ago around this time, I was working on a take-home task for a Harvard PhD interview, while compulsively refreshing my inbox and checking my spam folder for updates.

For most of my life, I believed China’s National College Entrance Examination (NCEE), aka the Gaokao, was the most intense and stressful exam a student could face.^[gaokao] It is a decade-long, high-stakes funnel that shapes college options for millions of students, so when I decided to leave law school, my mom took it very hard. And on a personal level, that period was especially difficult for me due to a chronic, deeply damaging family situation.

Years later, applying to graduate school taught me that the stress can take a different form. Instead of a single, clearly defined test, it’s often the uncertainty that wears you down—slowly, persistently, and without a clear endpoint. You can do many things “right” and still feel like you’re guessing what matters. That’s not to diminish the NCEE. It is difficult, no question about it, because millions of students compete for a limited number of seats. At the same time, however, it is transparent: you know what subjects you will be tested on, and once you clear the cutoff, the next step is usually straightforward.

Graduate admissions, at least in my experience, can feel like the opposite. It isn’t always obvious what is truly important, what is only sometimes important, or what you might be over-optimizing without realizing it. And if you’re applying from outside North America, that information gap can feel even wider.

❗Disclaimer: My experience is specific to applicants to EE/CS PhD programs in engineering-oriented areas such as systems and networking. Experiences can vary substantially across fields, departments, labs, and advisors.

My Results and Timeline

The table below summarizes the programs I applied to in the 2023–2024 cycle and the outcomes. The list is not in any particular order.

Two important notes:

• Many schools (e.g., Stanford and Princeton) allow applicants to apply to only one program per cycle, while others (e.g., MIT and CMU) allow applications to multiple programs in the same cycle.
• Although advising is often flexible across departments at most schools, different departments can have distinct policies. For example, at Princeton, ECE students are admitted by the department. They complete a rotation period in their first year and are matched with advisors afterward. In contrast, COS students are admitted directly into advisors’ labs. Rotation can be a great way to explore fit, but it can also be a stressful process for many.

What Actually Matters (and What Doesn’t)

I applied from Europe, and the information gap was one of the hardest parts. I did not always know what to prioritize, what was “nice to have,” and what was mostly noise. Most questions I get are about this, so I will start here.

Detailed recommendation letters

Rec letters are the most important part of the application.

Most North American programs will ask for 3–5 letters, and the application portal will ask whether you want to waive your right to read them. My strong recommendation is to waive access (select “Yes”). In practice, committees often treat waived letters as more candid and therefore more credible.

What matters even more than “prestige” is specificity. A detailed, sincere letter from a professor who worked closely with you can carry more weight than a vague letter from a famous name. Strong letters do more than list achievements. They explain what you contributed, how you think, how you collaborate, and how you show up as a teammate and researcher.

At some schools (like UofT and MIT), there is also an internal pre-screening process (for example, by a committee consisting of selected students) before applications reach individual faculty. In those settings, detailed letters can be a major differentiator because they provide concrete evidence of research readiness, independence, and reliability.

Do grades really matter?

Grades matter, but in most PhD admissions processes, they are not the primary signal.

If you are early in undergrad and considering a PhD, my advice is to treat coursework as important for building fundamentals while recognizing that research experience usually carries the most weight. Many faculty will look at grades as a supporting signal, and only lean on them if they lack other ways to assess your readiness.

There are also practical issues if you are applying internationally. Different grading systems can be hard to interpret. During one interview at the University of Toronto, a professor told me candidly that he was not sure how to evaluate my grades because they were on the Swiss 1–6 scale.

Most importantly, coursework and research can feel like different worlds. Even very demanding classes do not always translate into the skills needed to thrive in open-ended research. For more on my perspective, see my mentorship statement.^[mentorship]

Publications and research experience

The main reason to join a lab before applying is to build genuine research experience, and publications are one of the clearest ways to demonstrate that experience. A top-tier publication can completely change how your application is perceived.

At the same time, I do not believe publication count is a pure measure of talent. Timing, mentorship, project selection, and luck matter. Some students have early access to well-scoped problems and strong research pipelines; others are doing equally good work but have fewer structural advantages. Unfortunately, admissions often uses (first-author) publications as a proxy for research training, so they can matter a lot in practice.

This is also where I have mixed feelings. I personally believe a PhD should function more like an apprenticeship, and I will come back to this when I explain how I made my final decision.

ℹ️ As Prof. Peter Henderson opined, the bar for incoming graduate students has been raised too high, and the diversity of the students that clear it has started to collapse …

Faculty often look for similar traits

One surprising thing I noticed during admitted-student visits (I will write about it later) is that many of the visiting students overlapped across schools, making us like a huge traveling group. Over time, it became clear that different programs often admit students with similar/same profiles and signals.

Because of that, it can be helpful to study patterns. I recommend reading a range of statements of purpose and application materials.^{[sop-collection] [sop-mit]} Focus on:

• what kinds of research experiences they had
• what kinds of projects they worked on (and with whom)
• whether they had publications (and at what venues)
• what skills and framing show up repeatedly

For a structured discussion of how PIs and committees evaluate candidates, Chapter 2 of The CS Assistant Professor Handbook ^{[asstprofbook]} is a great resource. Also, MIT has a helpful page summarizing what faculty look for in application essays.^{[faculty-hints]}

Before the Actual Applications

Beyond the information gap, a second challenge (especially if you are applying internationally) is that you may have fewer informal channels for advice, mentorship, and introductions. That makes your timeline and outreach strategy even more important.

Who makes the admissions decision?

You’ll often hear people describe two admissions models: “professor-centered” admissions and “committee-based” admissions. In the former, individual professors/principal investigators (PIs) have significant influence over admissions decisions; in the latter, a committee made up of multiple faculty members (and sometimes students) plays a larger role.

That said, regardless of the formal structure, faculty preferences almost always matter the most. The main reason is funding: unless a student brings external funding that fully covers their PhD (often referred to as a “free student”), their support typically comes from a faculty member’s research grants.

To avoid substantial over- or under-hiring, departments usually ask faculty to report in advance how many students they expect to be able to fund in the upcoming admissions cycle. As a result, if one or more faculty members are genuinely interested in working with you and have the resources to support you, your chances of admission are very strong.

Start early and reach out

I started reaching out to professors and interviewing in May 2023, roughly seven months before most application deadlines. My first interview was on May 18 with Cornell, and it ended up being one of four interviews I had with Cornell faculty.

Here are three reasons to start early:

1. You learn about programs faster and assess fit sooner.
   To find a good match for your interests and skills, you need to understand departments and people in detail. This was especially important for me because I was applying from Europe. Some schools combine departments (for example, EECS), while others separate them (CS/CIS vs. EE/ECE). If your background sits between hardware and software systems, multiple program labels can be relevant, but the actual culture and focus vary widely.

   Even within “ECE,” departments can look very different. For example, ECE at UIUC and Cornell has a strong computing focus, while Princeton’s ECE spans a wide range of areas including VLSI, networking, AI, materials, bio, photonics, and quantum. So if you are interested in optical computing, ECE at Princeton might be a good fit, while at another school you might need to look at materials or applied physics departments instead.

2. You learn who is realistically hiring.
   Often, only a handful of faculty are a strong fit and have capacity in a given cycle. Several of my withdrawals happened because I learned faculty were not taking new students. For example, professors at UW and Columbia told me they were not able to take additional students due to recent over-hiring and the need to convert existing thesis students. That kind of information is hard to infer from their websites, and it matters even more during uncertain funding periods.

3. You build interview skills through repetition.
   Many faculty interviews include a short research presentation and deep technical questions. Starting early gave me time to improve my slides and, more importantly, my ability to explain my work under pressure.

   I also learned some lessons the hard way. I had a Yale interview scheduled jointly with multiple professors, and I struggled to answer rapid-fire questions before the conversation moved on. My slides also had issues that were easy for someone experienced to spot. It was not a great outcome, but it was a valuable learning experience. After that, I scheduled interviews one-on-one when possible, and I improved quickly. As a result, I later received offers from every program that interviewed me (except Yale, which is fine :P). So, you might want to schedule some early interviews with less competitive programs to prime yourself for the harder ones later.

:warning: There could be a hidden cost to reaching out early. Those early interviews can feel like a higher bar than the standard in-cycle ones, closer to “prove it now” than “let’s explore fit.” As Prof. Mae Milano noted: Cold outreach sometimes lands with faculty who are naturally skeptical and will quickly test whether there is a strong research match and clear readiness for the work. In those interviews, the questions can be more pointed and the expectations less scaffolded, which can feel sharper and less forgiving than a typical in-cycle interview.

The Interviews

The table below summarizes how many interviews I had and the types of interview tasks I encountered. I ranked programs based on the overall difficulty of the interviews and tasks (the higher the more difficult), but this is purely my personal experience (sample size == 1). Your experience may differ substantially.

A common question is why I sometimes had as many as five interviews at the same university. The reason is that multiple professors can be interested in the same candidate, and they usually don’t coordinate closely. So you might be interviewed by several faculty members independently.

As you can see, the most common tasks are:

• a short research presentation (often with slides)
• technical discussion
• and sometimes reading and reviewing papers

So it helps a lot to practice explaining your work clearly, and to get comfortable discussing papers critically and constructively.

Research presentation and technical discussion

Many interviews start with a short research presentation. A typical target is about 15 minutes for a structured overview of your work.

If you have papers, it helps to include a small number of key figures and focus on the core idea, the design choices, and the evaluation logic. You should also expect interruptions and questions, especially around trade-offs, alternatives, and “why did you do it this way?”

The most useful advice I can share is to proactively anticipate questions and practice your answers. Many questions repeat across interviews once you know what faculty tend to probe.

You may also interview with senior PhD students or postdocs. I often found those conversations especially engaging and sometimes more technically detailed. Students and postdocs may also be thinking about day-to-day collaboration, so they are assessing whether you would be a good teammate for the long run. If you know who you will meet, it is worth reading the group’s recent papers and skimming relevant code repos ahead of time.

Paper review

Here are a few example reviews I wrote for the University of Toronto ^{[reviews-uoft]} and Harvard interviews.^{[reviews-harvard]}

In these tasks, you are often evaluated on whether you can:

• accurately identify the paper’s core contributions and assumptions
• offer a thoughtful critique (strengths and limitations)
• propose concrete extensions or alternative approaches

Since the papers often come from the interviewing group, PIs may often be looking for whether you can develop a coherent “next step” that builds on their work.

A helpful starting guide is Prof. Onur Mutlu’s paper review guide: mutlu-guide

Coding and other tasks

Some programs have other types of tasks. For example, several systems faculty at the University of Toronto gave me coding tasks and take-home projects; the University of Cambridge asked for a 1K-word research proposal.

If you are curious, here is my codebase for a UofT coding interview task on implementing an SCMP ring buffer.^[ringbuffer] The repo includes both my implementation and experiment results.

School Visits

At the time, I lived in Zurich, Switzerland, and flights to the US were long and expensive. Fortunately, all US programs on my list (except UIUC) reimbursed travel and arranged hotels for me.

Visits are typically 2–3 days, and scheduling conflicts are common. For example, Cornell’s visit overlapped with Princeton and Penn’s, so I coordinated with Cornell to arrange an individual visit.

Caveat: weather can influence your experience

This is not a scientific point, but it is real. For instance, when I visited Boston, the weather was a bit dismal (cold and rainy), and while the city is wonderful, it affected how much I enjoyed being out and exploring. Similarly, Ithaca in late March was snowing like no tomorrow …

By contrast, Princeton had beautiful weather during my visit, and combined with the campus, it left a strong impression:

How I Made My Final Decision

People still ask me why I chose Princeton, especially since I invested a lot of time interviewing with other programs and faculty.

It wasn’t an easy decision. In fact, it was difficult enough that I ended up researching the science of making hard decisions and even made a video^{[decision-talk]} about it, which some of my friends found helpful:

Looking back, three factors mattered most.

1. I did not fully enjoy how “job-like” many interviews felt.
   I understand why selection is necessary as competition is fierce, but parts of the process conflicted with a core belief of mine: a PhD should be an apprenticeship. If applicants are expected to already have every skill needed to succeed in the treacherous PhD journey, it raises the question of what kind of mentorship and training the program is offering.

   This contrast stood out because, at the time, I also had a job offer from a big tech. I was pursuing a PhD specifically because I wanted a different kind of learning and growth.

   Surprisingly, I initially thought my Princeton interview went poorly. It turn into a high-intensity research discussion: I never got to present the slides that I had prepared and practiced a million times by then, and instead I was met with lots of probing questions that required real-time reasoning. When it ended, I couldn’t gauge how I had done, in part because the PI (now my advisor) kept a neutral expression throughout. Later, I learned that this is simply how she engages when she’s thinking deeply.

   I believe she saw potential in me without putting me through a slew of tests and checks. I am deeply grateful for that trust.

2. The people and the environment mattered a lot.
   During visits, I especially enjoyed meeting my current group members. They were consistently kind, supportive, and thoughtful, and I could imagine building a good working life with them. Culture is hard to measure from the outside, but it is one of the most important factors once you are actually doing the PhD.

   Additionally, after visiting many schools, I’ve come to believe that Princeton can satisfy all my imagination of a prototypical American campus, more so than anywhere else. After all, the word “campus” originated at Princeton.^{[campus-word]}

3. I wanted to lean into a new research direction.
   Most of my offers were in software/hardware systems, except for my advisor’s area. I appreciated that she welcomed me even though I did not have deep prior background in the specific subfield I would work on in my PhD.

   I’ve always believed that early in one’s career, it’s important to seize opportunities to try something different before settling into a specific field. This perspective is also echoed in this TED talk: david-talk

ℹ️ While financial considerations shouldn’t be the primary factor in choosing a PhD program, they matter a great deal in practice. Constantly worrying about making ends meet can seriously detract from your ability to focus on research. For this reason, I strongly recommend checking stipend levels alongside local cost of living before making a decision. For reference, here is a ranking of CS/EE PhD stipends in the US.^{[phd-stipends]}

Outro

Time truly flies, and it has now been over a year since I started my PhD at Princeton.

Did I make the right choice? I’d say there is no single “right” choice in the abstract; you make it right by what you build from it. What I can say is that, ever since starting, I’ve been enjoying every single day of my PhD journey. I’m grateful for my advisor, the engaging research, and the wonderful people around me.

Is it still worth pursuing a CS/EE PhD in the age of AI? That derserves a separate blog post. But if you want to do a PhD for money or status, do not. Your PhD journey is for you and only yourself. And if you do choose it, doing a PhD in a place like Princeton is an absolute privilege, and in my experience, a real joy ~

Acknowledgments

I want to thank Prof. Mae Milano, Seungju Lee, Constantine Doumanidis, Mathew Madain, Peilin Xin (CMU), Khánh Vũ, and Fengshi Zheng for their helpful comments and feedback on an earlier draft of this post. I’m also grateful to Prof. Keith Winstein for sending his advice to me as a reference.^{[winstein-advice]}

Resources

• Advice posted by Prof. Keith Winstein under the “Writing” section.^{[winstein-advice]}
• The CS Assistant Professor Handbook.^{[asstprofbook]}
• A collection of CS PhD statements of purpose.^{[sop-collection]}
• MIT CommLab guide on writing statements of purpose.^[sop-mit]
• Paper review guide by Prof. Onur Mutlu.^{[mutlu-guide]}
• CS/EE PhD stipend rankings (US).^{[phd-stipends]}
• MIT EECS page on what faculty look for in application essays.^{[faculty-hints]}
• My talk on making hard decisions.^{[decision-talk]}
• My UofT interview paper review examples.^{[reviews-uoft]}
• My UofT take-home coding project.^[ringbuffer]
• My Harvard interview paper review examples.^{[reviews-harvard]}

What Is Queuing Theory, and Why Should You Care?

In computer systems, latency is usually waiting, not doing. Requests wait for CPU time, for a DB connection, for a disk, for a lock, for a thread pool, for a downstream dependency, etc. Queuing theory is the toolkit for reasoning about that waiting:

• When does latency “blow up” as load increases?
• Which component is the bottleneck?
• How many workers/threads/servers do we need to hit an SLO?
• Why do averages look fine while p99/p999 gets ugly?

A useful mental model from systems performance practice is the “utilization knee”: as utilization approaches 100%, queues grow superlinearly and tail latency spikes.

The Basics: How Queues Map to Computer Systems

A queuing system has three parts:

1. Customers: requests, jobs, packets, RPCs, tasks.
2. Servers: CPU cores, threads, DB connections, GPU slots, NICs, disks.
3. Queue/buffer: ready queue, thread pool backlog, socket accept queue, message broker topic, request backlog at a load balancer.

A minimal “single service station” view of many components:

arrivals (λ)  --->  [ queue ]  --->  server(s) with rate μ  --->  departures

Key performance metrics you typically care about:

• Utilization: how busy the servers are.
• Queue length: how much work is waiting.
• Waiting time: how long work sits before service starts.
• Response time (sojourn time): waiting + service.
• Throughput: completed work per unit time.

Kendall’s Notation: A Handy Shorthand

Kendall’s notation is a concise way to name common models: A/S/C.

Examples:

• M/M/1: Poisson arrivals, exponential service, 1 server.
• M/M/c: same, but $c$ parallel servers (also called Erlang-C). ^[mmc-queue]
• G/G/1: “anything goes” arrivals and service times. ^[gg1-queue]

Queue discipline (FIFO, priority, shortest-job-first, processor-sharing) also matters for tail latency and fairness. ^{[queueing-theory]}

Essential Formulas (With Toy Computer-Systems Examples)

1. Little’s Law (works shockingly often)

• $L$: average number of requests in the system (waiting + in service)
• $\lambda$: effective arrival rate (throughput in steady state)
• $W$: average time in system (response time)

Little’s Law is extremely general: it does not require Poisson arrivals or exponential service—just a stable, long-run average system. ^{[littles-law]}

Example: “How many requests are in-flight?”

You run a service doing $1000$ requests/sec steady-state. Average end-to-end latency is $50$ ms.

• $\lambda = 1000$ req/s
• $W = 0.050$ s
• $L = \lambda W = 1000 \times 0.050 = 50$

So, on average, ~50 requests are concurrently in-flight across your service (queued + running). That’s often directly actionable:

• If your service uses one DB connection per in-flight request, you will need O(50) connections (plus headroom).
• If your service has a max in-flight limit of 30, you will either throttle to ~600 req/s or push queuing upstream.

2. Utilization and the stability condition

For $c$ identical servers with service rate $\mu$ each: \(\rho = \frac{\lambda}{c\mu}\)

Interpretation: $\rho$ is the fraction of time the server capacity is busy (on average).

Critical insight: if $\rho \ge 1$, the queue is unstable (it grows without bound). In real systems, “without bound” manifests as timeouts, retries, OOMs, thread exhaustion, and cascading failure.

Example: single-threaded handler

A single-threaded worker processes requests with mean service time 10 ms.

• $\mu = 1/0.010 = 100$ req/s
• If arrivals are $\lambda = 80$ req/s, then $\rho = 0.8$ (stable)
• If arrivals rise to $\lambda = 110$ req/s, then $\rho = 1.1$ (unstable)

This is why “CPU at 100%” is often a crisis: you have no slack to absorb randomness (bursts, long requests, GC pauses).

3. M/M/1: the simplest queue with a sharp lesson

For M/M/1, mean response time is: \(W = \frac{1}{\mu - \lambda}\) and mean waiting time in queue: \(W_q = W - \frac{1}{\mu}\)

Example: an API server with one worker thread

Same service time: 10 ms $\Rightarrow \mu = 100$ req/s.

If $\lambda = 80$ req/s:

• $W = 1/(100-80) = 1/20 = 0.05$ s = 50 ms
• Service time is 10 ms, so
• $W_q = 50 - 10 = 40 ms$ waiting
• By Little’s Law, $L = \lambda W = 80 \times 0.05 = 4$ requests in system, on average

Operational takeaway: Even at 80% utilization, average response time is 5× the service time (because most time is waiting). As $\rho \rightarrow 1$, $W$ diverges rapidly—this is the “latency knee.”

4. M/M/c (Erlang-C): adding workers reduces waiting—sometimes dramatically

For an M/M/c queue, the probability an arrival must wait (all servers busy) is given by the Erlang-C formula (often written in terms of offered load $a=\lambda/\mu$ and utilization $\rho=\lambda/(c\mu)$). ^[mmc-queue]

Rather than stare at the algebra, use it to answer a capacity question.

Example: sizing a thread pool / worker pool

Suppose:

• Arrival rate: $\lambda = 200$ req/s
• Each worker averages 12.5 ms per request $\Rightarrow \mu = 80$ req/s per worker

Case A: 3 workers

• $\rho = 200/(3 \times 80) = 0.833$

Erlang-C results (for these numbers):

• $P(\text{wait}) \approx 0.70$
• Mean queueing delay $W_q \approx 17.6$ ms
• Mean response time $W \approx 30.1$ ms

Case B: 4 workers

• $\rho = 200/(4 \times 80) = 0.625$

Now:

• $P(\text{wait}) \approx 0.32$
• $W_q \approx 2.7$ ms
• $W \approx 15.2$ ms

Interpretation: Adding one worker (3 → 4) did not increase capacity by 33% just to get 33% more throughput. It cut mean latency roughly in half by pulling utilization away from the knee and collapsing the queue.

This is a recurring theme in real services: once you are near saturation, small capacity changes can create huge latency changes.

5. Variability matters (a lot): the VUT / Kingman approximation

Real workloads are rarely “memoryless exponential everything.” Variability in arrivals (bursts) and service (slow queries, GC, cache misses) is what inflates tail latency.

A common, practical approximation for G/G/1 is Kingman’s formula (often called the VUT equation): ^{[kingman-formula]} \(\mathbb{E}[W_q] \approx \left(\frac{\rho}{1-\rho}\right)\left(\frac{c_a^2 + c_s^2}{2}\right)\tau\)

• $\rho$: utilization
• $c_a$: coefficient of variation of inter-arrival times
• $c_s$: coefficient of variation of service times
• $\tau$: mean service time

Example: same mean service time, different variability

Let:

• $\rho = 0.8$
• $\tau = 10$ ms
• arrivals roughly Poisson $\Rightarrow c_a \approx 1$

Scenario 1: highly variable service ($c_s = 1$)

• $\mathbb{E}[W_q] \approx (0.8/0.2) \times ((1^2+1^2)/2) \times 10 \text{ ms}$
• $= 4 \times 1 \times 10 = 40$ ms

Scenario 2: more predictable service ($c_s = 0.2$)

• $\mathbb{E}[W_q] \approx 4 \times ((1 + 0.04)/2) \times 10$
• $= 4 \times 0.52 \times 10 = 20.8$ ms

Same mean utilization and mean service time, but halving variability roughly halves queueing delay.

Operational takeaway: performance work that reduces variance (caching, eliminating stop-the-world pauses, bounding query time, isolating noisy neighbors) often improves tail latency more than shaving a millisecond off the mean. ^{[kingman-formula]}

Queues in Real Computer Systems: Patterns You’ll Actually See

1) A queue at every boundary

Common “queue boundaries” include:

• Load balancer backlog (requests waiting to be accepted)
• Thread pool queue (waiting for a worker)
• DB connection pool (waiting for a connection)
• Downstream service (RPCs queued behind other callers)
• Disk / network (I/O queues)

A key practice: apply Little’s Law per boundary to estimate concurrency needs and identify bottlenecks. ^{[littles-law]}

2) Bottlenecks dominate end-to-end latency

If a request must traverse multiple queues (a small queueing network), the slowest / most utilized stage tends to dominate waiting. Queueing networks are a standard lens for multi-tier services. ^{[queueing-theory]}

3) Queue discipline changes p99 behavior

FIFO is fair-ish, but it lets one slow job delay many fast ones. Alternative disciplines:

• Priority: protect latency-sensitive traffic (but risk starving background jobs). ^{[queueing-theory]}
• Shortest-job-first / SRPT: improves mean latency; may be unfair without safeguards. ^{[queueing-theory]}
• Processor sharing: approximates time-slicing, common in some network and CPU models. ^{[queueing-theory]}

This is why “just add a queue” is not neutral; it encodes policy.

Practical Modeling Workflow (Minimal but Useful)

1. Pick the boundary you’re modeling (e.g., DB connection pool).

2. Measure:

   • $\lambda$: arrival/throughput into that boundary
   • $\tau$: mean service time
   • variability proxies: p50/p95/p99 service times; burstiness of arrivals
3. Compute utilization $\rho$ and check headroom.

4. Use a simple model first:

   • M/M/1 or M/M/c for “first-order” capacity decisions. ^[mmc-queue]
   • Kingman (G/G/1) when variability is clearly the story. ^{[kingman-formula]}
5. Validate against production (does predicted $L$ match observed in-flight? does predicted knee match when p99 explodes?).

The goal is not perfect fidelity; it is a defensible mental model that prevents naive mistakes (like running everything at 95–99% utilization and being surprised by timeouts).

Limitations (What the Formulas Don’t Tell You)

• Non-stationarity: diurnal traffic, deployments, incident conditions.
• Correlated arrivals: synchronized retries, fan-out bursts, cache stampedes.
• Heavy-tailed service times: long-tail queries or rare slow paths can dominate p99/p999.
• Finite buffers and backpressure: real queues drop, throttle, or shed load.
• Retries change $\lambda$: a timeout often creates more work, not less.

Queueing theory still helps here, but you often combine it with measurement, tracing, and load testing.

Wrapping It All Up

If you remember only a few points:

• Little’s Law turns latency and throughput into concurrency: $L=\lambda W$. ^{[littles-law]}
• Utilization is destiny: near 100%, queues (and tail latency) explode.
• Variability inflates waiting: controlling variance can be more valuable than improving the mean. ^{[kingman-formula]}
• Adding capacity near the knee can cut latency disproportionately (Erlang-C intuition). ^[mmc-queue]

References

• Little’s Law (background). ^{[littles-law]}
• System performance and queueing intuition (talk). ^[srecon24]
• M/M/c (Erlang-C) model (background). ^[mmc-queue]
• Queueing theory in practice for performance modeling (talk). ^[lisa17]
• Kingman’s approximation / VUT equation (background). ^{[kingman-formula]}
• Harchol-Balter, M. (2013). Performance Modeling and Design of Computer Systems: Queueing Theory in Action. Cambridge University Press.
• Gross, D., & Harris, C. M. (1998). Fundamentals of Queuing Theory. John Wiley & Sons, Inc.
• Kleinrock, L. (1975). Queueing Systems, Volume 1: Theory. John Wiley & Sons, Inc.
• Sutton, C., & Jordan, M. I. (2010). Bayesian inference for queueing networks and modeling of internet services. ^{[sutton-jordan2010]}

In Linux, both processes and threads are assigned numeric identifiers, and you can see them show up as peer directories under the /proc pseudo-filesystem. Each schedulable entity appears as a subdirectory in the form /proc/[pid], where that number is often referred to as a “PID”.

Here is the catch: the value that shows up in /proc/[pid] is not strictly a process identifier. Depending on context, it may refer to a thread or a process, because Linux historically built threads on top of processes.

You can observe this terminology overload in tools like htop. By default, htop lists both processes and threads without clearly separating them, so its “PID” column contains identifiers that can correspond to either.

This has a historical reason. Early Linux did not have a first-class notion of threads, only processes. Over time, Linux introduced “thread groups” (Linux 2.4, around 2001), which support the POSIX threads model: multiple threads that conceptually belong to one process. Internally, the shared “process ID” that user space expects is implemented as a thread group identifier (TGID). As described in the clone(2) manual, getpid(2) returns the TGID of the caller, not a per-thread identifier.

As a result, threads in the same process share the same TGID, while each thread also has its own unique thread ID (TID). Practically, getpid() returns the same value across all threads in the process, while gettid() returns a unique value per thread.

How to Distinguish Between a Thread and a “Real Process”

Conceptually, Linux processes and threads are similar because the kernel schedules both as runnable entities. The major difference is what they share: threads in the same process typically share an address space and other resources, while separate processes usually do not (unless explicitly arranged).

To distinguish “threads within a process” from a standalone process, the most reliable place to look is:

• /proc/[pid]/task/[tid]

The task/ directory enumerates kernel-visible threads. The tid component is the kernel thread ID. This is distinct from user-level threading abstractions (for example, some managed runtimes can implement user-level “threads” that are not one-to-one kernel threads). Those user-level threads are not directly visible as separate kernel thread IDs.

Within a multithreaded process, all threads belong to the same thread group. The main thread has tid == tgid, and the other threads have distinct tids but the same tgid. You will also notice that /proc/[pid]/task/[tid] mirrors /proc/[pid]/ when pid == tid, because that path is effectively describing the same main thread and the same thread group leader.

So, when you inspect /proc/[pid] for a multithreaded process, you can interpret it as follows:

• The directory name /proc/[pid] corresponds to the process’s TGID (and the main thread’s TID).
• The /proc/[pid]/task/ subdirectories enumerate all kernel threads in that thread group.
• The “process” you intuitively think of is the thread group leader (the main thread) that created the other threads.

Last Note: Multiprocessing vs Multithreading

To avoid confusion with thread groups (TGID), it helps to also remember process groups, which use PGID. TGID is about threads within a process, while PGID is about groups of processes (used heavily for job control in shells).

In broad strokes, a new process is created via fork() (and friends), while a new thread is created via pthread_create() in C. Under the hood, Linux commonly uses the clone() syscall for both, with different flags controlling what is shared.

When a process spawns subprocesses, the spawning process is the parent, and it may either create a new process group or inherit an existing one. If it creates a new process group, the PGID is typically set to the PID of the process that created the group. If it inherits a group, the PGID will match the inherited group.

In contrast, the main thread and the threads it creates are best thought of as siblings under the same thread group: they share a TGID, and they also share the same PGID as the main thread. One practical implication is that threads are not visible as child “processes” to the parent of the thread group leader.

Example

A convenient way to generate a multithreaded workload on Linux is to use stress-ng. In stress-ng terminology, a “stressor” (or “hog”) is a process.

For example, the following command runs a memory contention stressor:

hy@node-0:~$ stress-ng --mcontend 1 -t 10h
stress-ng: info:  [56472] dispatching hogs: 1 mcontend

If you open htop, you can view the resulting process and its threads as a hierarchy. In that display, PGRP corresponds to the process group ID (PGID), while the PID column is overloaded and may represent either process IDs or thread IDs depending on the row.

In the example shown:

• 56472 is the single-threaded parent process created by your shell when you ran the command.
• 56473 is the multithreaded child process (and also the main thread of that child).
• 56474 through 56477 are sibling threads created by the main thread 56473.

If you run pidof on the stressor, you will typically get the TGID (which matches the main thread’s ID):

hy@node-0:~$ pidof stress-ng-mcontend
56473

You can confirm that 56472 is the parent of 56473 by inspecting the parent’s children file:

hy@node-0:~$ cat /proc/56472/task/56472/children
56473

Next, if you list the child’s task/ directory, you will see all kernel threads in that thread group:

hy@node-0:~$ ll /proc/56473/task/
total 0
dr-xr-xr-x 7 hy hy 0 Dec 31 22:21 ./
dr-xr-xr-x 9 hy hy 0 Dec 31 22:21 ../
dr-xr-xr-x 7 hy hy 0 Dec 31 22:21 56473/
dr-xr-xr-x 7 hy hy 0 Dec 31 22:21 56474/
dr-xr-xr-x 7 hy hy 0 Dec 31 22:21 56475/
dr-xr-xr-x 7 hy hy 0 Dec 31 22:21 56476/
dr-xr-xr-x 7 hy hy 0 Dec 31 22:21 56477/

If you inspect the task/ directory for one of the sibling threads, you will still see the full set of thread IDs in the group (because you are still within the same thread group context):

hy@node-0:~$ ll /proc/56476/task/
total 0
dr-xr-xr-x 7 hy hy 0 Dec 31 22:21 ./
dr-xr-xr-x 9 hy hy 0 Dec 31 22:21 ../
dr-xr-xr-x 7 hy hy 0 Dec 31 22:21 56473/
dr-xr-xr-x 7 hy hy 0 Dec 31 22:21 56474/
dr-xr-xr-x 7 hy hy 0 Dec 31 22:21 56475/
dr-xr-xr-x 7 hy hy 0 Dec 31 22:21 56476/
dr-xr-xr-x 7 hy hy 0 Dec 31 22:21 56477/

Finally, notice that the single-threaded parent process has only itself under task/:

hy@node-0:~$ ll /proc/56472/task/
total 0
dr-xr-xr-x 7 hy hy 0 Dec 31 22:21 ./
dr-xr-xr-x 9 hy hy 0 Dec 31 22:21 ../
dr-xr-xr-x 7 hy hy 0 Dec 31 22:21 56472/

Resource Accounting

Once you start looking at threads, resource accounting becomes another place where Linux tooling can be surprising. Different tools choose to aggregate or split resource usage in different ways, and sometimes they report thread-level information only when explicitly asked.

For example, htop typically aggregates the CPU and memory usage of all threads into the main thread’s row by default. Similarly, ps will usually show aggregated usage for the thread group leader when you query it by PID:

hy@node-0:~$ ps -p 56473 -o %cpu,%mem,cmd
%CPU %MEM CMD
 473  0.0 stress-ng-mcontend

If you try the same query on a sibling thread ID, you may get nothing because ps is often oriented around process identifiers unless you request thread detail:

hy@node-0:~$ ps -p 56476 -o %cpu,%mem,cmd
%CPU %MEM CMD

To display threads, you can use ps -L with the main thread’s ID:

hy@node-0:~$ ps -L 56473 -o %cpu,%mem,cmd
%CPU %MEM CMD
97.3  0.0 stress-ng-mcontend
94.0  0.0 stress-ng-mcontend
94.0  0.0 stress-ng-mcontend
94.0  0.0 stress-ng-mcontend
94.0  0.0 stress-ng-mcontend

If you want a more detailed listing, ps -L ... -F includes fields such as LWP (the thread ID) and NLWP (the number of threads):

hy@node-0:~$ ps -L 56473 -F
UID          PID    PPID     LWP  C NLWP    SZ   RSS PSR STIME TTY      STAT   TIME CMD
hy         56473   56472   56473 97    5 22792  2604  13 08:10 pts/2    RLl+ 302:30 stress-ng-mcontend
hy         56473   56472   56474 94    5 22792  2604   7 08:10 pts/2    RLl+ 292:03 stress-ng-mcontend
hy         56473   56472   56475 94    5 22792  2604  31 08:10 pts/2    RLl+ 292:00 stress-ng-mcontend
hy         56473   56472   56476 94    5 22792  2604  15 08:10 pts/2    RLl+ 291:59 stress-ng-mcontend
hy         56473   56472   56477 94    5 22792  2604   0 08:10 pts/2    RLl+ 292:05 stress-ng-mcontend

Thread reporting in top has similar behavior. With -H, top shows per-thread CPU usage, while without it, top aggregates:

hy@node-0:/proc$ top -H -p 56476
....
    PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND
  56473 hy        20   0   91168   2708   2272 R  97.3   0.0 127:24.91 stress-ng-mcont
  56474 hy        20   0   91168   2708   2272 R  94.0   0.0 122:55.16 stress-ng-mcont
  56475 hy        20   0   91168   2708   2272 R  94.0   0.0 122:54.44 stress-ng-mcont
  56476 hy        20   0   91168   2708   2272 R  93.7   0.0 122:55.33 stress-ng-mcont
  56477 hy        20   0   91168   2708   2272 R  92.3   0.0 122:56.57 stress-ng-mcont

Without -H, top reports a single aggregated number and attributes it to the thread group leader:

hy@node-0:~$ top -p 56476
....
    PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND
  56473 hy        20   0   91168   2708   2272 R 476.3   0.0 621:39.36 stress-ng-mcont

If you are scripting, you can use batch mode to extract either aggregate or per-thread CPU values, depending on whether you include -H:

# Aggregated
hy@node-0:~$ top -b -n 2 -d 0.2 -p 56476 | tail -1 | awk '{print $9}'
465.0

# Per-thread with -H
hy@node-0:~$ top -b -H -n 2 -d 0.2 -p 56476 | tail -1 | awk '{print $9}'
75.0

If you need ground-truth per-thread accounting, /proc is again the most explicit source. A thread’s /proc/[pid]/stat can reflect aggregated values when accessed via certain paths, whereas /proc/[pid]/task/[tid]/stat is the per-thread view:

# Total CPU time (user and kernel) aggregated across the thread group.
hy@node-0:~$ cat /proc/56476/stat | awk '{print $14, $15}'
9460932 12361

# CPU time for only thread 56476.
hy@node-0:~$ cat /proc/56476/task/56476/stat | awk '{print $14, $15}'
1879429 3032

You can also use psutil for a convenient scripting interface, but it is important to know what it aggregates. In many cases, psutil reports CPU and memory usage in a way that effectively attributes thread-group totals to whichever thread you query, because it is fundamentally process-centric:

>>> import psutil

# Great-grandparent process (e.g., tmux session).
>>> tmux_session = psutil.Process(54711)
>>> tmux_session.ppid()
1
>>> [(child.name(), child.pid) for child in tmux_session.children(recursive=True)]
[('bash', 54712), ('bash', 56236), ('python', 56613), ('stress-ng', 56472), ('stress-ng-mcontend', 56473)]

# Parent process.
>>> parent = psutil.Process(56472)
>>> parent.ppid()
54712
>>> parent.children(recursive=True)
[psutil.Process(pid=56473, name='stress-ng-mcontend', status='running', started='11:21:57')]
>>> parent.num_threads()
1

# Child process (main thread / thread group leader).
>>> child = psutil.Process(56473)
>>> child.num_threads()
5
>>> [thread.id for thread in child.threads()]
[56473, 56474, 56475, 56476, 56477]

# Sibling thread example.
>>> sibling = psutil.Process(56476)
>>> child.ppid()
56472
>>> sibling.ppid()
56472

# Accounting.
>>> parent.cpu_percent(interval=1)
0.0
>>> sibling.cpu_percent(interval=1)
471.5
>>> child.cpu_percent(interval=1)
472.4

>>> tmux_session.cpu_times()
pcputimes(user=7.46, system=3.19, children_user=102.18, children_system=153.15, iowait=0.0)
>>> parent.cpu_times()
pcputimes(user=0.0, system=0.0, children_user=0.0, children_system=0.0, iowait=0.0)
>>> child.cpu_times()
pcputimes(user=45250.11, system=57.79, children_user=0.0, children_system=0.0, iowait=0.0)
>>> sibling.cpu_times()
pcputimes(user=45255.42, system=57.79, children_user=0.0, children_system=0.0, iowait=0.0)

>>> parent.memory_full_info()
pfullmem(rss=6475776, vms=59777024, shared=6078464, text=1728512, lib=0, data=32018432, dirty=0, uss=3051520, pss=3749888, swap=0)
>>> child.memory_full_info()
pfullmem(rss=2772992, vms=93356032, shared=2326528, text=1728512, lib=0, data=65581056, dirty=0, uss=126976, pss=735232, swap=0)
>>> sibling.memory_full_info()
pfullmem(rss=2772992, vms=93356032, shared=2326528, text=1728512, lib=0, data=65581056, dirty=0, uss=126976, pss=735232, swap=0)

>>> tmux_session.memory_percent()
0.007239506814671662
>>> parent.memory_percent()
0.009602063988251591
>>> child.memory_percent()
0.004111699759675096
>>> sibling.memory_percent()
0.004111699759675096

The upshot is that “what is the PID” and “what resources belong to a thread” depend strongly on which abstraction and which tool you are using. Many tools default to aggregating at the thread-group level, even when they print thread IDs, and you typically need explicit flags (or direct /proc inspection) to get consistent per-thread views.

To end the running example, sending an interrupt to one thread can terminate the entire stressor, depending on how the program handles signals:

# Note: this returns True even if the process is a zombie.
>>> parent.is_running() == child.is_running() == sibling.is_running() == True
True

>>> import signal
>>> sibling.send_signal(signal.SIGINT)

>>> parent.is_running() == child.is_running() == sibling.is_running() == False
True

(It seems that interrupting one thread has a bottom-up cascading effect in stress-ng 🥴 )

Happy New Year 🎆 ~

Reference

• https://unix.stackexchange.com/questions/364660/are-threads-implemented-as-processes-on-linux
• https://unix.stackexchange.com/questions/670836/why-do-threads-have-their-own-pid
• https://stackoverflow.com/questions/1221555/retrieve-cpu-usage-and-memory-usage-of-a-single-process-on-linux
• https://unix.stackexchange.com/questions/404054/how-is-a-process-group-id-set
• https://stackoverflow.com/questions/4856255/the-difference-between-fork-vfork-exec-and-clone
• https://stackoverflow.com/questions/19678954/relation-between-thread-id-and-process-id
• https://stackoverflow.com/questions/9430491/find-cpu-usage-for-a-thread-in-linux
• https://stackoverflow.com/questions/1420426/how-to-calculate-the-cpu-usage-of-a-process-by-pid-in-linux-from-c
• https://stackoverflow.com/questions/19919881/sysconf-sc-clk-tck-what-does-it-return
• https://www.baeldung.com/linux/total-process-cpu-usage
• https://psutil.readthedocs.io/en/latest/#processes
• https://man7.org/linux/man-pages/man5/proc.5.html
• https://man7.org/linux/man-pages/man2/getpid.2.html
• https://man7.org/linux/man-pages/man2/gettid.2.html
• https://manpages.ubuntu.com/manpages/bionic/man1/stress-ng.1.html
• https://www.akkadia.org/drepper/nptl-design.pdf

• time until a server fails
• time until a customer churns
• time until an incident is resolved
• time until a job completes
• time until a cache entry expires (or until a key becomes “cold”)

What makes survival analysis special is that it handles incomplete observation windows correctly. In real engineering datasets, you frequently do not observe the event for everyone before you stop collecting data:

• A customer hasn’t churned yet when you export the dataset.
• A disk hasn’t failed yet when you stop the experiment.
• A request hasn’t completed yet when you cut off a trace.

Treating those as “missing” or dropping them biases you toward shorter times and overconfident conclusions. Survival analysis is designed to keep that partial “last-seen” information and remain statistically consistent.

Survival data in systems: what you actually record

A time-to-event dataset typically has at least two columns:

• duration: how long you observed the unit (user/machine/request)
• event: whether the event occurred within your observation window (1) or not (0)

A concrete toy example: time-to-churn (days) for 6 trial users.

Key idea (from the reference video): the “event=0” rows are not “unknown.” They tell you the unit lasted at least that long. ^{[survival-video]}

You also need to be explicit about your time origin:

• For churn: trial start, first purchase, first session, last renewal?
• For failures: installation time, last maintenance, last reboot?
• For incidents: page time, incident creation time, first alert time?

Changing the origin changes the interpretation of the curves and coefficients.

Two core functions: survival and hazard (with intuition)

Survival function answers: “What fraction lasts beyond time t?”

For churn, you can read $S(30)=0.8$ as “80% of users are still active after 30 days.” ^{[survival-analysis]}

Hazard function answers a different question: “Given you’ve made it to time t, how ‘risky’ is right now?”

Formally:

In systems terms, hazard is often closer to what you want operationally:

• “Given a server has been up 20 days, what is its instantaneous failure rate now?”
• “Given a user is still active at week 4, what is the instantaneous churn pressure now?”

A practical relationship to remember:

• high hazard around time t means the survival curve drops steeply around time t
• if hazard increases with time, you’re in “wear-out” behavior (common in hardware); if it decreases, you may be seeing “early-life failures” (infant mortality) or onboarding drop-off

A common derived quantity is the cumulative hazard:

This becomes useful when you want to add hazards over stages or interpret models on a log scale. ^{[survival-analysis]}

The Kaplan–Meier curve: a survival curve from observed events plus “last-seen” records

If you do not want to assume a distribution, the Kaplan–Meier (KM) estimator gives a non-parametric estimate of $S(t)$ using (a) observed event times and (b) “last-seen active” times for units that did not experience the event within the window. ^{[kaplan-meier]}

It is:

Where at each event time $t_i$:

• $n_i$ = number “at risk” just before $t_i$ (still being observed, event not yet happened)
• $d_i$ = number of events at $t_i$

Toy KM computation (churn example).
Using the 6-user table above, sort by time and update the risk set:

Event times are 2, 3, 4, 6 (and we have “last-seen” times at 7 and 10).

• At day 2: $n=6$, $d=1$ → multiply by $(1-1/6)=5/6$
• At day 3: $n=5$, $d=1$ → multiply by $(1-1/5)=4/5$
• At day 4: $n=4$, $d=1$ → multiply by $(1-1/4)=3/4$
• At day 6: $n=3$, $d=1$ → multiply by $(1-1/3)=2/3$

So:

• $\hat S(2)=5/6 \approx 0.833$
• $\hat S(3)=(5/6)(4/5)=4/6 \approx 0.667$
• $\hat S(4)=(5/6)(4/5)(3/4)=3/6 = 0.5$
• $\hat S(6)=(5/6)(4/5)(3/4)(2/3)=2/6 \approx 0.333$

Where “last-seen” matters: users B and D remain “at risk” up to days 7 and 10, respectively, contributing correctly to $n_i$ up to those times. KM is effectively saying: “they didn’t churn before 7/10, and that information counts.” ^{[kaplan-meier]}

A highly practical workflow for product/systems:

• Plot KM curves for cohorts (e.g., different regions, hardware batches, onboarding variants).
• Look for when curves separate: early vs late differences are operationally distinct (onboarding vs retention; infant mortality vs wear-out).

Comparing groups: the log-rank test (what it is and when it lies)

To test whether two groups have different survival curves, the log-rank test compares observed vs expected events over time under the null that the groups share the same survival distribution. ^{[logrank-test]}

The intuition:

• At each event time, if Group A has 60% of the risk set, then under the null it “should” get ~60% of the events.
• The log-rank statistic aggregates how much reality deviates from that expectation across time.

When it is useful:

• quick sanity check that a cohort difference is not noise
• comparing two versions of a system or product feature rollout

When it can mislead:

• if the curves cross (effects change over time)
• if observation windows differ systematically between groups (e.g., one region has shorter follow-up windows)

Cox proportional hazards: turning covariates into “risk multipliers”

The Cox proportional hazards model connects covariates to hazard without specifying the baseline hazard shape. ^[cox-ph]

How to read it in practice:

• $\exp(\beta_j)$ is a hazard ratio for a 1-unit increase in $X_j$.
• Hazard ratio > 1 means “riskier” (event happens sooner on average), < 1 means “protective.”

Toy interpretation (churn).
If your Cox model yields:

• $\exp(\beta_{\text{annual_plan}})=0.7$

Then, holding other covariates fixed, annual-plan users have 30% lower instantaneous churn hazard than monthly-plan users at any given time.

The key assumption: proportional hazards means those hazard ratios are roughly constant over time (i.e., the curves differ by a multiplicative factor in hazard, not by shape). When this is false, you may need:

• time-varying covariates/effects
• stratified Cox
• an accelerated failure time (AFT) model for “time scaling” instead of “hazard scaling” ^{[survival-analysis]}

Practical checklist for engineers

1) Define observation windows explicitly
Write down, in the experiment doc (not just in code), what “duration” means and when observation stops (end of study, export date, decommission time, trace cutoff).

2) Check whether observation stopping is correlated with the event
Ask simple pipeline questions:

• Do some users “disappear” because tracking stops (not because behavior changed)?
• Do some machines leave the dataset because they were proactively replaced (possibly due to warning signs)?

If “stopping observation” is correlated with the event, naive analyses can bias survival upward.

3) Start with KM curves before fitting models
KM gives shape intuition: early drop, long tail, crossing hazards, etc. ^{[kaplan-meier]}

4) Use Cox when you need covariate-adjusted answers
Examples:

• adjust failure risk for load, temperature, and batch
• adjust churn risk for acquisition channel and engagement

5) Operationalize outputs
Good time-to-event analysis ends with a decision:

• “Which cohort should we target, and when?”
• “What burn-in period reduces infant mortality?”
• “Which covariate is the strongest early-warning signal?”

A simple operational trick: evaluate $S(t)$ at business-relevant horizons (e.g., day 1, day 7, day 30) and report those, not just “the curve.”

References

• Survival analysis (overview). ^{[survival-analysis]}
• Kaplan–Meier estimator. ^{[kaplan-meier]}
• Log-rank test. ^{[logrank-test]}
• Proportional hazards / Cox model. ^[cox-ph]
• Introduction to survival data and incomplete observation windows (video). ^{[survival-video]}

In many cases, it’s better to trade a bit more bias for a much smaller variance to tighten generalisation error estimates. One way to do so is by adding a regularizer. In linear regression, L1 (Lasso) and L2 (Ridge) regularization are two common options. Lasso suppresses small weights to zero, making the feature space sparse, while Ridge shrinks all weights. Both restrict the norm of the weights and help mitigate overfitting.

However, regularization is only one way to strike the balance. Another way is to introduce more bias into the equation through the Bayesian lens. Specifically, we can impose prior knowledge by adding a prior distribution to constrain the norm of the leant parameters. For example, if we know the weights are small and centred, then we can set our prior to be $\vec{w} \sim \mathcal{N}(0, \beta\textbf{I})$. Then, by Bayes rule, we have:

Thus, both regularization and Bayesian modelling can achieve the same goal, which begs the question: are they connected?

Laplace is to Lasso as Gaussian is to Ridge

The answer to the above question turns out to be yes! To illustrate this further, let’s use two common prior distributions, Lapace and Gaussian, as our running examples.

Firstly, we assume the following general setting for regression: ${y} = X{\theta}$ and $f_X = y + \epsilon$ where $\theta \sim \text{Laplace}(0, s) = 1/2s \cdot\exp(-\mid\theta\mid / s)$ and $\epsilon \sim \mathcal{N}(0, \delta^2_\epsilon)$.

Then, we obtain the maximum a posteriori (MAP) esitmation as:

Next, we can substitute both the likelihood and prior into Eq. (1).

where $Z$ is the Gaussian normalising constant. By simplifying Eq. (2), we obtain the following form:

Now, we have recovered the exact form of Lasso, where $\cfrac{2\delta^2_\epsilon}{s}$ is the coefficient of the L1 regularizor $\lambda$ that controls the strength of the constraint.

Next, let’s play the same trick in the same setting but with a Gaussian prior instead, i.e., $\theta \sim \mathcal{N}(0, \delta_\theta^2)$.

Starting from Eq. (1), we subsitute in the likelihood and prior as above:

Finally, letting go all the fluff in Eq. (5), we have:

By Eq. (6), we have recovered Ridge regression where the fraction $\cfrac{\delta_{\epsilon}^2}{\delta_{\theta}^2}$ denotes regularization constant $\lambda$.

Summary

By working out the above two examples, we found that regularised regression is nothing but Bayesian modelling in disguise. In fact, imposing various priors has the same effect as using corresponding regularizers. By the same token, choosing different likelihoods gives us different loss functions. In this post, we used Gaussian likelihood in both examples, and, in turn, recovered the square loss.

There are many other options for prior and likelihood functions. For instance, one can use a student-t as opposed to Gaussian. Lastly, a family of conjugate prior can drastically reduce the cost of Bayesian inference.

This is also a rolling log of books I recently read (2021) that are right up my alley.

Bennett, Arnold. Self and self-management: Essays about existing. George H. Doran Company, 1918.

The true stories behind the stories of success shall be the same and might not be as glorious.

Hawking, Stephen. The theory of everything. Jaico Publishing House, 2006.

Well, I’m now under the impression that computer science is a pseudoscience :}

Hall, Herbert James. The untroubled mind. Houghton Mifflin, 1915.

How can I live out a life so fully that worries couldn’t sneak in? Perhaps most importantly, what’s my deeper justification and higher pursuit thereof?

Jones, Diana Wynne, John Sessions, and Stella Paskins. Howl’s moving castle. Recorded Books, 2008.

I finally understand why I didn’t quite understand the movie 😅

Some of my friends from China saw a bunch of blanks here. This is because YouTube videos can’t pass the firewall 🧱.

10 ways to have a better conversation

• If you want to pontificate, go write a blog.
• Ask open-ended questions.
• “No man ever listened his way out of a job” — Calvin Coolidge
• Most of us don’t listen with the intent to understand. We listen with the intent to reply.
• Listen, and be prepared to be amazed.
• If you don’t know, say that you don’t know.
• Don’t equate your experience with others.

Self-control

This Ted talk conveys very similar messages as that of the below one. Their main ideas are covered in many other talks as well.

To reach beyond your limits by training your mind

• Self-control is the problem where we have all these desires from ourselves for the long-term, but then in the short-term, we do rather different things (that prevent ourselves from achieving long-term those goals).
• Our will power is weak, and therefore, it’s not something upon which we should rely during decision-making.
• If we are faced with temptation whilst having no tool at hand to overcome it, we’re almost certainly going to fail.
• We should collaborate with our brains with constructive messages.
  • Change the pictures and the words. Using very detailed words.
  • Tell your mind exactly what you want.
• We ought to create tools that will control our future selves to do what our current selves want them to do.
  • It’s a situation where we know we will be tempted, and we do something to make ourselves not be able to be tempted.
• Make the familiar unfamiliar and the unfamiliar familiar.
• Reward-substitution: connecting pain to pleasure
  • Link massive pleasure to going there and pain to not going there.
  • Do the right thing for the wrong reason.
• Make Self Belief so normal to you that everyone believes in you too.

The last point ties in with another Ted talk, which will be introduced later.

Body language, the power is in the palm of your hands

• The palm up position is friendly and inviting, whilst the palm down position exerts power and control over others.
  • Finger pointing is the worst; it is directive and rude.

Imposters: The psychology of pretending to be someone you’re not

• Some imposters are escapologists — they’re running away from a flawed past and trying to rehabilitate their image.
• We spend a hell of a lot of time trying to impress other people.
• Ultimately, life is a performance with all the sense of drama, anarchy, and possibility.
• Who ever controls the past controls the future; who ever controls the present controls the past. — Orwell

How to draw to remember more

• Thinking in pictures and then drawing them down is a great way to learn and memorize new things.
• There are a myriad of ways to represent an abstract concept using drawings.
• The quality of the drawing does not matter at all. In other words, good and bad pictures have rather similar, if not the same, effect on the learning process.
• It is the process of doing the drawing that actually makes a difference.

To be continued 👨‍💻 …

Step 1: key pair generation

Use the following command to generate a public (silver) / private (black) RSA key pair under the ~/.ssh/id_rsa directory. The .ssh/id_rsa is the private key you keep on your machine, and .ssh/id_rsa.pub is the public key you distribute to other machines to enable passwordless logins.

ssh-keygen -t rsa

Note that git uses a different type of cryptosystem, namely, the Ed25519 system, which can be generated using the following command.

ssh-keygen -t ed25519 -C "youremail@yourdomain"

Compared to RSA, it is considered to be faster, safer and more compact (Ed25519: 8chars, RSA: 544chars) although RSA is more commonly used.

Step 2: distribute key(s)

If Alice wants to log in Server1 shown in the figure, she can run the following command to forward her SSH public key(s) to it. The keys sent will be recorded in .ssh/authorized_keys of the host.

ssh-copy-id alice-username@server1.domain-or-ip

Afterwards, Alice should be able to log in Server1 without being asked for her password. I.e.,

ssh alice-username@server1.domain-or-ip

Welcome to XXX ...

Note that for Mac users that do not have ssh-copy-id, you can either intall it via brew or mannually copy these ssh files through scp, rsync or whatnot. If you go for the latter, one thing you should keep in mind is to set the permission bits correctly as shown below.

chmod 700 ~/.ssh
chmod 600 ~/.ssh/*

Step 3: indirect login

In the case that Server1 has a firewall or what have you, Alice has to connect it via a proxy, say Server2; therein lies the question: how to access Server2 directly using key-pair authentication?

To tackle this, Alice can first forward her/his keys to Server2, the proxy, through Step 2. Next, Alice should log in Server2 to generate a key pair (Step 1), and then, send keys to Server1.

Last, but certainly not least, Alice should set up her ssh on her local machine. The configuration (~/.ssh/config) is along the lines of the following.

Host server2
    HostName          server1-domain
    User              alice-username
    IdentityFile      ~/.ssh/id_rsa

Host server1
    HostName          server1-domain
    User              alice-username
    ForwardX11Trusted yes
    ForwardAgent      yes
    IdentityFile      ~/.ssh/id_rsa
    ProxyCommand ssh server2 -W %h:%p

Now, everything should be in place. Alice should be able to do login, port forwarding, or whatnot, with automatic key-pair authentication (without having to type her password every single time for every single server along the way!).

🛑 NB: Data loads between Server1 and Server2 is not encrypted.

# Log in *Server1* via *Server2*.
ssh server1 

# Tunnelling ports to *Server1* via *Server2*.
ssh -NL {listening_port}:{hostmachine}:{host_port} server1 

p.s. Wrestling with company proxies during these COVID times we live in at the moment can be devastating 😷

The main idea of autoencoders is to extract latent features that are not easily observable yet play an important role in one or several aspects of the data (e.g., images).

1.1 Compression (by the “encoder”)

The first step of the process is to compress the observed data vector $\vec x$ into the latent feature vector $\vec z$.

There are two obvious benefits yielded from such compression process:

1. The latent feature vector $\vec z$ is much smaller, which makes it much easier to process than the original (potentially high-dimensional) data.
2. As its name suggests, the latent feature vector $\vec z$ may capture important hidden features. 

1.2 Reconstruction (by the “decoder”)

The second phase is to try to reproduce the data (the image) from the latent feature vector $\vec z$. 

Apparently, since the first step is a “lossy compression”, the data reconstructed $\vec{\hat{x}}$ will not be exactly the same as the original observation. Here is where the third phase comes about.

1.3 Backpropagation

As mentioned above, there is a difference between the observation $\vec{x}$ and the reconstruction $\vec{\hat{x}}$.

From the above picture, we can see clearly that the higher the dimension of the latent feature vector $\vec{z}$, the higher the quality of the reconstruction.

Therefore, constraining the size of the latent space will enforce the “importance” of the extracted features. 

Further, we can use a loss function to measure such “importance” of the extracted hidden variables. In this case, we use a simple square loss:

Thus, the key power of autoencoders is that

Autoencoder allows us to quantify the latent variables without labels (gold-standard data)!

To summarize, 

• Autoencoding == Automatically encoding data
• Bottleneck hidden layer forces the network to learn a compressed latent representation.
• Reconstruction loss forces the latent representation be as “paramount” and “informative” as possible.

2 Variational Autoencoders (VAEs)

2.1 Stochastical variation

In a nutshell, variational autoencoders are a probabilistic twist on autoencoders, i.e. (stochastically) sample from the mean and standard deviation to compute the latent sample as supposed to deterministically take the entire latent vector $\vec{z}$. That been said, the main idea of the forward propagation does not change compared to traditional autoencoders. 

• In the compression process, the encoder computes $p_{\phi}(\mathrm{z} \mid x)$.
• In the reconstruction phase, the decoder computes $q_{\theta}(\mathrm{x} \mid z)$.

Then, we could compute the loss as follows

which is exactly the same as before. It captures the pixel-wise difference between the input and the reconstructed output. This is a metrics of how well the network is doing at generating the distribution that akin to that of the observation.

As to the “regularization term”, since the VAE is producing these probability distributions, we want to place some constraints on how they are computed as well as what that probability distribution resembles as a part of regularizing and training the network.

Hence, we place a prior $p(z)$ on the latent distribution as follows

which captures the KL divergence between the inferred latent distribution and this fixed prior for which a common choice is a normal Gaussian, i.e. we centre it around with a mean of 0 and a standard deviation 1: $\ p(z)=\mathcal{N}\left(\mu=0, \sigma^{2}=1\right)$.

In this way, the network will learn to penalise itself when it tries to cheat and cluster points outside sort of this smooth Gaussian distribution as it would be the case if it was overfitting or trying to memorize particular instances of the input.

Thus, this will enforce the extracted $\vec z$ follows the shape of our initial hypothesis about the distribution, smoothing out the latent space and, in turn, helping the network not over-fit on certain parts of the latent space.

2.2 Backpropagation? Reparametrization

Unfortunately, due to the stochastic nature, the backpropagation cannot pass the sampling layer as backpropagation requires deterministic nodes to be able to iteratively pass gradients and apply the chain rule through.

Instead, we consider the sampled latent vector $\vec z$ as a sum of a fixed vector $\vec \mu$ a fixed variance vector $\vec \sigma$ and then scaled this variance vector by a random constant that is drawn from a prior distribution, for example from a normal Gaussian. The key idea here is that we still have a stochastic node but since we have done this reparametrization with the factor $\epsilon$ that is drawn from a normal distribution, this stochastic sampling does not occur directly in the bottleneck layer of $\vec z$. This way, we can reparametrize where that sampling is occurring.

Note that this is a really powerful trick as such reparametrization is what allows for VAEs to be trained end-to-end.

3 Code example

The following is a vanila implementation of a VAE model in Tensorflow.

class Sampling(keras.layers.Layer):
  def call(self, inputs) :
  z_mean, z_log_var = inputs
  batch = tf.shape(z_mean)[0]
  dim = tf.shape(z_mean)[1]
  epsilon = tf.keras.backend.random_normal_(shape=(batch, dim))
  return z_mean + tf.exp(0.5 * z_log_var)

latenet_dim = 2
encoder_inputs = Input(shape=(6), name="input_layer")

X = Dense (5, activation="relu", name="h1")(encoder_inputs)
X = Dense (5, activation="relu", name="h2")(x)
X = Dense (4, activation="relu", name="h3")(x)
z_mean = Dense(latent_dim, name="z_mean")(x)
z_log_var = Dense(latent_dim, name="z_log_var")(x)
z = Sampling()([z_mean, z_log_var])

encoder = keras.Model(encoder_inputs, [z_mean, z_log_var, z], name="encoder")

keras.utils.plot_model(encoder, show_shapes=True)