Welcome

This is my personal space where I keep all my notes from my ML experience. I’m mostly interested in NLP and graph embeddings/analytics. Thus expect most of the material to cover:

Fundamentals of Deep Learning;
Fundamentals of Reinforcement Learning;
Basic Stats;
Basic of Neural ODE.

Please note that this website is ment to keep only my personal material, this it my contains errors and typos.

Here are some working principles I try to apply during my working expeience.

My vision of Skunkworks Rules

The Skunk Works workers must be delegated practically complete control of his program in all aspects.
The number of people having any connection with the project must be restricted in an almost vicious manner. Use a small number of good people (10% to 25% compared to the so-called normal systems).
A very simple drawing and drawing release system with great flexibility for making changes must be provided.
There must be a minimum number of reports required, but important work must be recorded thoroughly.
Access by outsiders to the project and its personnel must be strictly controlled by appropriate security measures.
Because only a few people will be used in engineering and most other areas, ways must be provided to reward good performance by pay not based on the number of personnel supervised.

Musk’s Development Principles

Make the requirements less dumb. “Your requirements are definitely dumb, it does not matter who gave them to you.” Uses this step to test assumptions and question-the-question, especialy if it come from a ‘smart person’ since you might not question them enough.
Delete the part or process. “If you’re not adding things back in at least 10% of the time, you’re clearly not deleting enough.” Starting lean and building up when and if required, but be strongly bias throward removing thinks. Each requirement or constraint must be accountable to a person, so you can ask that person about its relevance and purpose.
Simplify or optimise the design. “Possibly the most common error of a smart engineer is to optimise a thing that should not exist”. Do your work and go through the first two steps before trying to optimise. To do this effectively you need an holistic view of the project and you need to consider the overall cost-benefit for a more complicated design.
Accelerate cycle time. “Try a bunch of things, and do more of what works”. Once you have find the right direction is time to accelerate the lifecycle. Aim for an experiment a day. Note that, it is extreamly importanto to do the first 3 steps before as “if you’re digging your grave, don’t dig faster.”

Not To Do

In 1944, the CIA wrote a handbook on how to sabotage (enemy) organizations from the inside. A few interesting tips:

Insist on doing everything through “channels.” Never permit short-cuts to be taken in order to expedite decisions.
Make “speeches.” Talk as frequently as possible and at great length.
When possible, refer all matters to committees, for “further study and consideration.” Attempt to make the committees as large as possible—never less than five.
Bring up irrelevant issues as frequently as possible.
Haggle over precise wordings of communications, minutes, resolutions.
Refer back to matters decided upon at the last meeting and attempt to re-open the question of the advisability of that decision.
Demand written orders.
“Misunderstand” orders. Ask endless questions or engage in long correspondence about such orders. Quibble over them when you can.
Do everything possible to delay the delivery of orders. Even though parts of an order may be ready beforehand, don’t deliver it until it is completely ready.
In making work assignments, always sign out the unimportant jobs first.
Insist on perfect work in relatively unimportant products; send back for refinishing those which have the least flaw. Approve other defective parts whose flaws are not visible to the naked eye.
When training new workers, give incomplete or misleading instructions.
To lower morale and with it, production, be pleasant to inefficient workers; give them undeserved promotions. Discriminate against efficient workers; complain unjustly about their work.
Hold conferences when there is more critical work to be done.
Multiply paper work in plausible ways.
Start duplicate files.
Multiply the procedures and clearances involved in issuing instructions, pay checks, and so on. See that three people have to approve everything where one would do.
Apply all regulations to the last letter.
Do your work poorly and blame it on bad tools, machinery, or equipment. Complain that these things are preventing you from doing your job right.
Never pass on your skill and experience to a new or less skillful worker.
Snarl up administration in every possible way. Fill out forms illegibly so that they will have to be done over; make mistakes or omit requested information in forms.
Give lengthy and incomprehensible explanations when questioned.
Act stupid.
Be as irritable and quarrelsome as possible without getting yourself into trouble.
Misunderstand all sorts of regulations concerning such matters as rationing, transportation, traffic regulations.

Figure 1: Credit to Roel Staes linkedIn post.

Razor’s Rules

Everyone is falimialr with the infamous Occam’s razor rule. However, recelty I stuped uppon a George Mack tweet listing a bunch of other useful rules:

Bragging Razor: If someone brags about their success or happiness, assume it’s half what they claim. If someone downplays their success or happiness, assume it’s double what they claim
High Agency Razor: If unsure who to work with, pick the person that has the best chances of breaking you out of a 3rd world prison.
The Early-Late Razor: If it’s a talking point on Reddit, you might be early. If it’s a talking point on LinkedIn, you’re definitely late.
Luck Razor: If stuck with 2 equal options, pick the one that feels like it will produce the most luck later down the line. I used this razor to go for drinks with a stranger rather than watch Netflix. In hindsight, it was the highest ROI decision I’ve ever made.
Buffett’s Law; “The value of every business is 100% subject to government interest rates” - Warren Buffett
The 6-Figure Razor: If someone brags about “6 figures” – assume it’s closer to $100K than $900K.
Parent Rule: Break down the investments your parents made in you: Time, Love, Energy, and Money. If they are still alive, aim to hit a positive ROI (or at least break even.)
Instagram Razor: When you see a photo of an influencer looking attractive on Instagram – assume there are 99 worse variations of that photo you haven’t seen.They just picked the best one.
Narcissism Razor: If worried about people’s opinions, remember they are too busy worrying about other people’s opinions of them. 99% of the time you’re an extra in someone else’s movie
Everyday Razor: If you go from doing a task weekly to daily, you achieve 7 years of output in 1 year. If you apply a 1% compound interest each time, you achieve 54 years of output in 1 year.
Bezos Razor: If unsure what action to pick, let your 90-year-old self on death bed choose it.
Creativity Razor: If struggling to think creatively about a subject, transform it:
- Turn a thought into a written idea;
- A written idea into a drawing;
- A drawing into an equation;
- An equation into a conversation.
  In the process of transforming it, you begin to spot new creative connections.
The Roman Empire Rule: Historians now recognize the Roman Empire fell in 476 – but it wasn’t acknowledged by Roman society until many generations later. If you wait for the media to inform you, you’ll either be wrong or too late.
Physics Razor: If it doesn’t deny the law of physics, then assume it’s possible. Do not confuse society’s current lack of knowledge – with this knowledge being impossible to attain. E.g., the smartphone seems impossible to someone from the 1800s – but it was possible, they just had a lack of knowledge.
Skinner’s Law: If procrastinating, you have 2 ways to solve it:
- Make the pain of inaction > Pain of action;
- Make the pleasure of action > Pleasure of inaction.
Network Razor: If you have 2 quality people that would benefit from an intro to one another, always do it. Networks don’t divide as you share them, they multiply.
Gell-Mann Razor: Assume every media article contains a % of false information. Sandbox the article from your worldview until you’ve: 1) Seen primary sources; 2) Spoken to 3 domain experts.
Taleb’s Surgeon: If presented with two equal candidates for a role, pick the one with the least amount of charisma. The uncharismatic one has got there despite their lack of charisma. The charismatic one has got there with the aid of their charisma.