last updated: 2024-05-09

A curated compilation of research papers, tech blogs, and videos that I find most valuable and inspiring.

1/ Papers

1.1/ general

Zanzibar: Google’s Consistent, Global Authorization System design, implementation, and deployment of Zanzibar, a global system for storing and evaluating access control lists for Google services: Calendar, Cloud, Drive, Maps, Photos, and YouTube

Towards Modern Development of Cloud Applications Introduces modular monoliths.

Delta Lake: High-Performance ACID Table Storage over Cloud Object Stores The delta table format for big data lakehouse

Bigtable: A Distributed Storage System for Structured Data Google’s paper on the bigtable format

Presto: SQL on Everything SQL Query engine for interactive analytics in big data.

Napa: Powering Scalable Data Warehousing with Robust Query Performance at Google A customizable data warehouse technology for google

Dynamo: Amazon’s Highly Available Key-value Store Foundational paper on AWS DynamoDB

Gorilla: A Fast, Scalable, In-Memory Time Series Database A great paper which explains the core ideas behind time-series databases

Sharding the Shards: Managing Datastore Locality at Scale with Akkio Meta’s data placement service

AutoDev: Automated AI-Driven Development Autonomous Developer paper from Microsoft.

Google File System The google dile system paper that HDFS is based on

Meta’s Microservice Architecture HN Discussion Thread

1.2/ ai/ml

todo

2/ Blogs

2.1/ Aggregated links:

2.2/ Individual links:

How Meta built the infrastructure for Threads The infra that powers theads

How Levels.fyi scaled to millions of users with Google Sheets as a backend Frugal data storage using google sheets

Intelligent DNS based load balancing at Dropbox How dropbox makes its DNS faster

Working with PostgreSQL (Zerodha) How India’s largest stock broker used to use postgres

The technology behind GitHub’s new code search How does github’s code search work

Using Server Sent Events to Simplify Real-time Streaming at Scale Managing BFCM events using server sent events

#BehindTheBug — Indexing Gone Wrong How wrong indexing crashed Swiggy instamart

αcτµαlly pδrταblε εxεcµταblε Cross platform C/C++ binaries

How we built a general purpose key value store for Facebook with ZippyDB Meta’s distributed key/value store

Asynchronous computing @Facebook: Driving efficiency and developer productivity at Facebook scale Meta’s asynchronous compute platform

Managing data store locality at scale with Akkio Meta’s data placement service: Akkio

How we built Pingora, the proxy that connects Cloudflare to the Internet Cloudflare’s nginx replacement

Fixing duplicate stories in Medium’s For You feed How medium fixed duplicate stories

Designing a wide-column database on RocksDB How Pinterest built a wide-column database on RocksDB

2.3 / Others

Smashing the state machine: the true potential of web race conditions Web race condition attacks

3/ Videos

3.1/ Youtube Channels

todo

3.2/ Playlists

Linux Device Driver Videos Networking, Kernel, IPV6, Device Drivers

3.3/ Individual Videos

Linux packet journey,napi, hardware queue,skb

4/ Awesomness

System Design

awesome-scalability The Patterns of Scalable, Reliable, and Performant Large-Scale Systems

awesome-distributed-systems A curated list to learn about distributed systems

awesome-grpc A curated list of useful resources for gRPC

Graphs

awesome-graph A curated list of resources for graph databases and graph computing tools

awesome-explainable-graph-reasoning A collection of research papers and software related to explainability in graph machine learning.

Awesome-GFlowNets A curated list of resources about generative flow networks (GFlowNets).

Graph Neural Networks (GNN)

Data Engineering

awesome-data-engineering A curated list of data engineering tools for software developers

awesome-opensource-data-engineering An Awesome List of Open-Source Data Engineering Projects

awesome-flink A curated list of amazingly awesome Flink and Flink ecosystem resources

Cryptography / Blockchain / DLT

awesome-zero-knowledge-proofs A curated list of awesome things related to learning Zero-Knowledge Proofs (ZKP).

AI / ML

awesome-chatgpt-prompts A curated list of awesome Machine Learning frameworks, libraries and software.

awesome-machine-learning A curated list of awesome Machine Learning frameworks, libraries and software.

awesome-production-machine-learning A curated list of awesome open source libraries to deploy, monitor, version and scale your machine learning

awesome-deep-vision A curated list of deep learning resources for computer vision

awesome-self-supervised-learning A curated list of awesome self-supervised methods

Awesome-LLMOps An awesome & curated list of best LLMOps tools for developers

Causal Inference/ML

Causal Inference/ML Link 1
Causal Inference/ML Link 2
Causal Inference/ML Link 3
Causal Inference Visual Guides
Awesome-Uplift-Model - Causal ML How to Apply Causal ML to Real Scene Modeling？How to learn Causal ML？

Meta Learning

Self supervised learning

Transformers

Quantum

awesome-quantum-machine-learning Here you can get all the Quantum Machine learning Basics, Algorithms ,Study Materials ,Projects and the descriptions of the projects around the web