Implement **causal attention masking** for a self-attention score matrix in NLP, so each token can only attend to itself and earlier tokens. You’ll take raw attention scores and apply a mask that forces “future” positions to have zero probability after softmax.

The masked attention is:
$$
A = \text{softmax}(S + M)
$$
where $S$ is the score matrix and $M_{ij} = 0$ if $j \le i$, otherwise $M_{ij} = -10^9$.

## Requirements

Implement the function
```python
def masked_softmax(scores):
    \"""
    Applies a causal (lower-triangular) mask to attention scores and then computes softmax.

    Args:
        scores (np.ndarray): Attention score matrix S of shape (n, n),
            where scores[i][j] is the raw score for token i attending to token j.

    Returns:
        np.ndarray: Attention probability matrix A of shape (n, n),
            where A[i][j] is the masked softmax probability.
    \"""
    # Your solution below
```

**Rules:**
- Apply a **causal mask** so positions where `j > i` are masked out (future tokens).
- Use a **numerically stable** softmax per row (subtract the row max before `exp`).
- Use `-1e9` (or another very large negative number) for masked positions before softmax.
- Return the result as a NumPy array.
- Use only NumPy (and built-in Python libraries if needed).

## Example

```python
scores = np.array([
    [1.0, 2.0, 3.0],
    [1.0, 2.0, 3.0],
    [1.0, 2.0, 3.0],
])

masked_softmax(scores)
```

Output:
```python
[
    [1.0, 0.0, 0.0],
    [0.2689, 0.7311, 0.0],
    [0.0900, 0.2447, 0.6652]
]
```

Projection onto box constraints

Longest Uncommon Subsequence II

Attention masking

Solve this medium level ML coding question from General. The premier LeetCode for ML platform to master Python, SQL, and Machine Learning algorithms.

624. Attention masking

Examples

Constraints

Similar Questions