data science
intermediate
10 sample questions
Neural Networks MCQ Practice Test
Study of artificial neural networks inspired by biological neural networks.
Q1. In a deep neural network trained for image classification using ReLU activation functions, you observe that a significant portion of neurons in the later layers consistently output zero during both training and inference. Which of the following is the MOST likely cause?
-
A. The learning rate is too high, causing weights to oscillate wildly.
-
B. The network is overfitting, focusing on noise in the training data.
-
C. The weights in these layers have become excessively large, pushing the input values outside the ReLU's positive range.
-
D. The weights in these layers have become excessively small, resulting in consistently negative inputs to the ReLU activation function. ✓
Explanation: The ReLU (Rectified Linear Unit) activation function outputs zero for negative inputs and the input value for positive inputs. If a significant portion of neurons consistently output zero, it suggests their inputs are consistently negative. This indicates that the weights connecting to these neurons are likely too small, pushing the weighted sum of inputs below zero, thus causing the ReLU to always output zero. While other options can contribute to poor network performance, this is the most direct explanation for the observed phenomenon of many consistently zero-valued neurons with ReLU activation.
Q2. What is the primary function of the activation function in a neural network?
-
A. It helps to prevent overfitting
-
B. It increases the non-linearity of the model ✓
-
C. It is used for regularization
-
D. It is a type of optimization algorithm
Explanation: The activation function is used to introduce non-linearity into the model, allowing it to learn complex relationships between inputs and outputs.
Q3. In a neural network with a sigmoid activation function, what is the output of the node when the input is 0?
-
A. The output is 0.5 ✓
-
B. The output is 0
-
C. The output is 1
-
D. The output is a probability of 0.5
Explanation: The sigmoid function maps any real-valued number to a value between 0 and 1. When the input is 0, the sigmoid function returns 0.5, which is a probability of 0.5.
Q4. In a neural network, what is the primary function of the "activation function" in a hidden layer?
-
A. It is used to weight and normalize the input features.
-
B. It introduces non-linearity to the model, allowing it to learn complex relationships. ✓
-
C. It is a type of regularization technique to prevent overfitting.
-
D. It is used to update the model's weights during backpropagation.
Explanation: The activation function is used to introduce non-linearity to the model, enabling it to learn complex relationships between inputs and outputs. This is essential for neural networks to model real-world problems, which often involve non-linear relationships.
Q5. In the context of neural networks, what is the primary purpose of the “weight decay” regularization technique?
-
A. To prevent overfitting by adding a penalty term to the loss function for large weights ✓
-
B. To improve generalization by reducing the impact of noise in the training data
-
C. To accelerate training by initializing weights with a small random value
-
D. To increase the capacity of the network by adding more hidden layers
Explanation: Weight decay is a regularization technique that adds a penalty term to the loss function for large weights. This helps to prevent overfitting by encouraging the model to learn smaller weights, which can lead to better generalization performance.
Q6. In the context of neural networks, what is the primary purpose of the “forgetting” mechanism in the synaptic plasticity rule known as “STDP (Spike-Time-Dependent Plasticity)”?
-
A. To stabilize the learning process by preventing overfitting
-
B. To enable the network to adapt to changing environments
-
C. To introduce a temporal component to synaptic learning, allowing the network to “forget” weak or irrelevant connections ✓
-
D. To improve the network's ability to generalize to new data
Explanation: STDP is a synaptic plasticity rule that allows neural networks to learn and adapt based on the timing of spikes. The forgetting mechanism in STDP enables the network to gradually weaken or eliminate connections that are no longer relevant or useful, thereby allowing it to adapt to changing environments and learn from new experiences.
Q7. In a deep neural network, what is the primary purpose of using a technique such as weight normalization in the context of batch normalization?
-
A. To improve the stability of the model by reducing the impact of weight initialization. ✓
-
B. To enable the model to learn more robust features by normalizing the weights across different layers.
-
C. To prevent the vanishing gradient problem by scaling the weights uniformly across all layers.
-
D. To accelerate the training process by reducing the number of parameters in the network.
Explanation: Weight normalization reparameterizes weights (direction/magnitude), decoupling learning from the initial weight scale and stabilizing training — especially alongside batch normalization.
Q8. A neural network is trained to classify handwritten digits using a convolutional neural network (CNN) architecture. The network is composed of multiple convolutional and pooling layers, followed by fully connected layers. However, the network is experiencing a phenomenon known as "feature dominance" where a single feature is dominating the classification decision. Which of the following techniques could be used to mitigate this issue?
-
A. Increase the number of convolutional layers to allow the network to learn more complex features.
-
B. Use a different pooling strategy, such as max pooling, to reduce the spatial dimensions of the feature maps.
-
C. Apply a regularization technique, such as dropout, to the fully connected layers to prevent any single feature from becoming too dominant. ✓
-
D. Use a technique called "feature attention" to selectively focus on the most relevant features for the classification decision.
Explanation: Feature dominance occurs when a single feature becomes too dominant in the classification decision, leading to poor performance on other features. Regularization techniques, such as dropout, can help to prevent this by randomly dropping out neurons during training, preventing any single feature from becoming too dominant.
Q9. In a deep neural network, what is the primary purpose of the ReLU (Rectified Linear Unit) activation function in the hidden layers?
-
A. To introduce non-linearity and enable the network to learn complex relationships between inputs and outputs ✓
-
B. To reduce the effect of outliers in the data
-
C. To improve the generalization of the model by preventing overfitting
-
D. To accelerate the convergence of the backpropagation algorithm
Explanation: The ReLU activation function is widely used in deep neural networks because it introduces non-linearity in the model, allowing it to learn complex relationships between inputs and outputs. This is in contrast to linear activation functions, which can only learn linear relationships.
Q10. In the context of neural networks, what is the primary purpose of the “forgetting” mechanism in the experience replay buffer used in Deep Q-Networks (DQN)?
-
A. to prevent overfitting by reducing the impact of outdated experiences ✓
-
B. to improve the stability of the target network by updating it more frequently
-
C. to increase the entropy of the policy by introducing random actions
-
D. to reduce the variance of the policy gradient estimate by downweighting older experiences
Explanation: The forgetting mechanism in the experience replay buffer is used to prevent the DQN from overfitting to outdated experiences. By downweighting older experiences, the network is forced to learn from a more diverse set of experiences, which helps to improve its generalization abilities.
That was just a sample. Sign up to unlock the full question bank with timed tests and certificates.
Sign Up Free