Abstract: In this talk, we will introduce our recently proposed deep neural networks for solving the photometric stereo problem for non-Lambertian surfaces. Traditional approaches often adopt simplified reflectance models and constrained setups to make the problem more tractable, but this greatly hinders their applications on real-world objects. We propose a deep fully convolutional network, named PS-FCN, that predicts a normal map of an object from an arbitrary number of images captured under different light directions. Compared with other learning based methods, PS-FCN does not depend on a pre-defined set of light directions and can handle multiple images in an order-agnostic manner. To tackle uncalibrated light directions, we propose a deep convolutional neural network, named LCNet, that predicts per-image light direction and intensity from both local and global features extracted from all input images. We analyze the features learned by LCNet and find they resemble attached shadows, shadings, and specular highlights, which are known to provide clues in resolving GBR ambiguity. Based on this insight, we propose a guided calibration network, named GCNet, that explicitly leverages object shape and shading information for improved lighting estimation. Experiments on synthetic and real data will be presented to demonstrate the effectiveness of our proposed networks.