This article investigates the problem of inverse optimal control (IOC) for a class of nonlinear affine systems. An adaptive IOC approach is proposed to recover the cost functional using only the system state data, which integrates the finite-time concurrent learning (FTCL) technique and the semidefinite programming (SDP) technique. First, an identifier neural network (NN) is employed to approximate the unknown nonlinear control policy, and an FTCL-based update law is proposed to estimate the weights of the identifier NN online, which removes the traditional persistent excitation (PE) condition. Moreover, the finite-time convergence as well as the uniformly ultimately boundness (UUB) of estimation error of the identifier NN weights are analysed according to whether or not there exists the identifier NN approximation error. Then, with the help of a value NN for approximating the value function, an SDP problem with a quadratic objective function can be set up for determining the weighting matrices of the cost functional. Finally, simulation results are presented to validate the proposed method.