Join Our 5-Week ML/AI Engineer Interview Bootcamp 🚀 led by ML Tech Leads at FAANGs

Back to Questions

234. Gradient clipping

easy
GeneralGeneral
senior

Implement global norm gradient clipping, a common trick in optimization to prevent exploding gradients during training. Given a list of gradient vectors, you’ll scale them down only if their combined (global) L2 norm is too large.

The clipping rule is:

scale=min⁡(1, max_norm∑i∥gi∥22),gi′=scale⋅gi\text{scale} = \min\left(1,\ \frac{\text{max\_norm}}{\sqrt{\sum_i \|g_i\|_2^2}}\right),\quad g_i' = \text{scale}\cdot g_i

Requirements

Implement the function

python

Rules:

  • Compute the global L2 norm across all gradients in grads (treat all entries as one long vector).
  • If the global norm is <= max_norm, return gradients unchanged (numerically, same values).
  • Otherwise, multiply every value in every gradient vector by a single shared scale.
  • Use NumPy for the math; don’t use any deep learning framework utilities.
  • Keep the output as a list of NumPy arrays.

Example

python

Output:

python
Input Signature
ArgumentType
gradslist
max_normfloat
Output Signature
Return NameType
valuelist

Constraints

  • Use NumPy; no deep-learning utilities.

  • Return list of np.ndarray.

  • Single shared scale for all gradients.

Hint 1

Compute the global norm by summing squares of all entries across every gradient vector, then take sqrt.

Hint 2

Use a single shared scale: scale = min(1.0, max_norm / global_norm) (handle global_norm == 0). Apply it to every element.

Hint 3

With NumPy: accumulate total_sq += np.sum(g**2) over vectors; global_norm = np.sqrt(total_sq); output must stay list of arrays.

Roles
ML Engineer
AI Engineer
Data Scientist
Quantitative Analyst
Companies
GeneralGeneral
Levels
senior
entry
Tags
gradient-clipping
L2-norm
numpy
optimization
32 people are solving this problem
Python LogoPython Editor
Ln 1, Col 1

Input Arguments

Edit values below to test with custom inputs

You need tolog in/sign upto run or submit