A

Step 1: Given that the refractive index of the glass slab is \( \mu = A - Bt \), we need to analyze how light travels in this medium.
Step 2: By Snell's law at the air-glass interface, \( n_a \sin \theta_a = \mu \sin \theta_g \), where \( n_a \) is the refractive index of air (approximately 1), \( \theta_a \) is the angle of incidence, and \( \theta_g \) is the angle of refraction in the glass slab.
Step 3: With the variation of refractive index with depth, it indicates that as light travels deeper, the speed of light in the glass decreases. This means that the angle \( \theta_g \) will change as well.
Step 4: As the thickness \( t \) increases, the refractive index decreases proportionally, reaching a limit where further increases in depth will lead to a reduction in light's effective path. To find the maximum depth the light can travel, we analyze the conditions where it approaches total internal reflection right before emerging back.
Step 5: Thus, the light ray travels to a depth given by the maximum angle to remain within the glass, calculated effectively using geometric relations and setting up limits under total internal reflection principles. The resultant maximum depth can be derived mathematically based on the refractive index formula and trigonometric properties, clearly leading us to the primary mathematical limit.
Therefore, the answer is option A.