Linkedin
Facebook
Youtube
Twitter
Instagram
Pinterest
Flipboard
RSS

About Global Advisors
Insights
Careers
News
Contact

Select Page

Global Advisors | Quantified Strategy Consulting

training

"Diffusion models are a class of generative artificial intelligence (AI) models that create new data instances by learning to reverse a gradual, step-by-step process of adding noise to training data." - Diffusion models -

Term: Diffusion models

"Model weights are the crucial numerical parameters learned during training that define a model's internal knowledge, dictating how input data is transformed into outputs and enabling it to recognise patterns and make predictions." - Model weights -

Term: Model weights

"A loss function, also known as a cost function, is a mathematical function that quantifies the difference between a model's predicted output and the actual 'ground truth' value for a given input." - Loss function -

Term: Loss function

"Synthetic data is artificially generated information that computationally or algorithmically mimics the statistical properties, patterns, and structure of real-world data without containing any actual observations or sensitive personal details." - Synthetic data -

Term: Synthetic data

"The scaling hypothesis in artificial intelligence is the theory that the cognitive ability and performance of general learning algorithms will reliably improve, or even unlock new, more complex capabilities, as computational resources, model size, and the amount of training data are increased." - Scaling hypothesis -

Term: Scaling hypothesis

"Since 2020, we have seen a 600 000x increase in the computational scale of decentralized training projects, for an implied growth rate of about 20x/year." - Jack Clark - Import AI

Quote: Jack Clark – Import AI

“Most of the infrastructure cost for AI is for inference: serving AI assistants to billions of people.” - Yann Lecun VP & Chief AI Scientist at Meta

Quote: Yann Lecun

Four critical DeepSeek enablers: Multi-Head Latent Attention (MLA) This mechanism enables the model to process multiple facets of input data simultaneously, improving both efficiency and performance Mixture-of-Experts DeepSeek employs an MoE system that activates only a subset of its total parameters during any given task FP8 Floating Point Precision DeepSeek-V3 utilizes FP8 floating point precision during training, which helps in reducing memory usage and accelerating computation ~800 000 Distilled RL Parameters Deepseek only takes ~800 000 samples of good RL-reasoning to convert other dense models to RL-reasoning Other LLM model builders are racing to recreate and incorporate DeepSeek's approach

Infographic: Four critical DeepSeek enablers

"The most surprising part of DeepSeek-R1 is that it only takes ~800k samples of 'good' RL reasoning to convert other models into RL-reasoners. Now that DeepSeek-R1 is available people will be able to refine samples out of it to convert any other model into an RL reasoner." - Jack Clark, Anthropic

Quote: Jack Clark

“DeepSeek-R1 is AI’s Sputnik moment.” Marc Andreessen Andreesen Horowitz

Quote: Marc Andreessen

“With R1, DeepSeek essentially cracked one of the holy grails of AI: getting models to reason step-by-step without relying on massive supervised datasets.” Jeffrey Emanuel Former professional investor

Quote: Jeffrey Emanuel

Download brochure

What we do, case studies and profiles of some of our amazing team.

Our latest podcasts on Spotify

Sign up for our newsletters - free

Email *

Lists

All GA content
Fast Facts Proprietary short-form and graphic content produced by the Global Advisors team
GA Thoughts Proprietary Global Advisors’ perspectives on a wide variety of business topics.
Strategy Tools Global Advisors’ thoughts on classic, new and proprietary strategy tools and their applications.
GA News Updates about our firm and team.
Business News AM Our selection of the top business news sources on the web.
Business Select PM A daily bite-size selection of top business content.

I agree to the privacy policy and terms. (Link)

Recent Posts

Term: Lorenz curve
Term: Cournot equilibrium
Quote: Zig Ziglar – American author
Term: Bitter lesson
Quote: John D. Rockefeller – American businessman and philanthropist

Recent Comments

Global Advisors on 2017 experienced hires
Anthony Emslie on 3,6% of South African retirement funds make up 80% of total value

Linkedin
Facebook
Youtube
Twitter
Instagram
Pinterest
Flipboard
RSS

Global Advisors | Quantified Strategy Consulting