Google Interview Hacks

Describe back-propagation. We can calculate the error of the network only at the output units. The hidden units represent latent variables; we cannot observe their true values in the training data and thus, we have nothing to compute their error against. In order to update their weights, we must propagate the network's errors backwards through its layers. We will begin with Output1. Its error is equal to the difference between the true and predicted outputs, multiplied by the partial derivative of the unit's activation. Continue this process all the way to the input variables and then forward propagate the updated weights through the network.

What are the two main types of artificial neural networks? 1) Feedforward neural networks are the most common type of neural net, and are defined by their directed acyclic graphs. Signals only travel in one direction—towards the output layer—in feedforward neural networks 2) Feedback neural networks, or recurrent neural networks, do contain cycles. The feedback cycles can represent an internal state for the network that can cause the network's behavior to change over time based on its input.

Describe the Multilayer perceptron (MLP) The multilayer perceptron (MLP) is the one of the most commonly used artificial neural networks. The name is a slight misnomer; a multilayer perceptron is not a single perceptron with multiple layers, but rather multiple layers of artificial neurons that can be perceptrons. The layers of the MLP form a directed, acyclic graph. Generally, each layer is fully connected to the subsequent layer; the output of each artificial neuron in a layer is an input to every artificial neuron in the next layer towards the output.

What are the three components to an Artificial Neural Network? 1) The model's architecture, or topology, which describes the layers of neurons and structure of the connections between them. 2) The activation function used by the artificial neurons. 3) The third component is the learning algorithm that finds the optimal values of the weights.

What is kernalization? Answer: Projects linearly inseparable data to a higher dimensional space in which it is linearly separable.

What is an epoch? Answer: Each pass through the training instances

What is the perceptrons update rule? Answer: w(t + 1) = w(t) + alpha (d_j - y_j(t))x_i_j, for all features 0 <= i <= n

What is an error-driven learning algorithm? If the prediction is correct, the algorithm continues to the next instance. If the prediction is incorrect, the algorithm updates the weights.

Describe the steps of PCA 1) The first step of PCA is to subtract the mean of each explanatory variable from each observation: 2) Next, we must calculate the principal components of the data. Recall that the principal components are the eigenvectors of the data's covariance matrix ordered by their eigenvalues. The principal components can be found using two different techniques. 3) Next, we will project the data onto the principal components. The first eigenvector has the greatest eigenvalue and is the first principal component. We will build a transformation matrix in which each column of the matrix is the eigenvector for a principal component. 4) Finally, we will find the dot product of the data matrix and transformation matrix.

What are the principal components of PCA? The principal components of a matrix are the eigenvectors of its covariance matrix, ordered by their corresponding eigenvalues. The eigenvector with the greatest eigenvalue is the first principal component; the second principal component is the eigenvector with the second greatest eigenvalue, and so on.

Describe which descriptive characteristic of an eigenvector changes when transformed by a vector A. The direction of an eigenvector remains the same after it has been transformed by A; only its magnitude has changed, as indicated by the eigenvalue; that is, multiplying a matrix by one of its eigenvectors is equal to scaling the eigenvector.

What describes a vector? A vector is described by a direction and magnitude, or length.

Describe principal component analysis. PCA reduces the dimensions of a data set by projecting the data onto a lower-dimensional subspace. In general, an n-dimensional dataset can be reduced by projecting the dataset onto a k-dimensional subspace, where k is less than n. More formally, PCA can be used to find a set of vectors that span a subspace, which minimizes the sum of the squared errors of the projected data. This projection will retain the greatest proportion of the original data set's variance. Each subsequent principal component preserves the maximum amount of the remaining variance; the only constraint is that each must be orthogonal to the other principal components. PCA is most useful when the variance in a data set is distributed unevenly across the dimensions.

Motivate the need for dimensionality reduction. Dimensionality reduction is motivated by several problems. First, it can be used to mitigate problems caused by the curse of dimensionality. Second, dimensionality reduction can be used to compress data while minimizing the amount of information that is lost. Third, understanding the structure of data with hundreds of dimensions can be difficult; data with only two or three dimensions can be visualized easily.

Describe a method to evaluate the clusters. The silhouette coefficient is a measure of the compactness and separation of the clusters. It increases as the quality of the clusters increase; it is large for compact clusters that are far from each other and small for large, overlapping clusters. The silhouette coefficient is calculated per instance; for a set of instances, it is calculated as the mean of the individual samples' scores. The silhouette coefficient for an instance is calculated with the following equation: s = (ba) / max(a,b) a is the mean distance between the instances in the cluster. b is the mean distance between the instance and the instances in the next closest cluster.

Describe K-Means Clustering. The K-Means algorithm is a clustering method that is popular because of its speed and scalability. K-Means is an iterative process of moving the centers of the clusters, or the centroids, to the mean position of their constituent points, and re-assigning instances to their closest clusters. The cost function sums the distortions of the clusters. Each cluster's distortion is equal to the sum of the squared distances between its centroid and its constituent instances. The distortion is small for compact clusters and large for clusters that contain scattered instances.

What's the difference between an eager-learner vs lazy-learner? Eager learners build the model and that takes a while but then classify quickly. Lazy-learners wait to do the majority of the work when classifying.

What are random forests? A random forest is a collection of decision trees that have been trained on randomly selected subsets of the training instances and explanatory variables. Random forests usually make predictions by returning the mode or mean of the predictions of their constituent trees; scikit-learn's implementations return the mean of the trees' predictions. Random forests are less prone to overfitting than decision trees because no single tree can learn from all of the instances and explanatory variables; no single tree can memorize all of the noise in the representation.

What are ensemble learning methods? Methods combine a set of models to produce an estimator that has better predictive performance than its individual components.

What is Gini Impurity? Gini impurity is a measure of how often a randomly chosen element from the set would be incorrectly labeled if it was randomly labeled according to the distribution of labels in the subset. Gini impurity can be computed by summing the probability {\displaystyle f_{i}} f_{i} of each item being chosen times the probability {\displaystyle 1-f_{i}} 1-f_{i} of a mistake in categorizing that item. It reaches its minimum (zero) when all cases in the node fall into a single target category.