Neural Network Foundations with Keras
Building the First Trainable Model
Layered Computation
A feedforward neural network applies a sequence of affine transformations and nonlinear activations. For a dense layer, the computation is typically , followed by an activation such as ReLU or softmax.
Core Keras Components
Common Activations
ReLU
- Simple and efficient
- Works well in hidden layers
- Can suffer from dead neurons
Softmax
- Produces class probabilities
- Used in final multi-class outputs
- Outputs sum to
Training Objective
Training tries to minimize loss by adjusting parameters so predictions better match the targets.
Dense Network Workflow
-
1
1. Define the input shape.
-
2
2. Stack dense layers with activations.
-
3
3. Choose a loss function for the task.
-
4
4. Select an optimizer such as Adam or SGD.
-
5
5. Compile the model and train it on batches.
What does the expression represent in a dense layer?
Dense layers first apply a linear transformation plus bias before passing results to an activation.
Correct answer: An affine transformation
Why are activations needed in neural networks?
Without nonlinear activations, stacked linear layers collapse into a single linear transformation.
Correct answer: To add nonlinearity
Which optimizer is commonly used as a strong default choice?
Adam is widely used because it adapts learning rates per parameter and works well in many settings.
Correct answer: Adam