We study the stochastic kinetics of Ca$^{2+}$ ion channels in the olfactory cilium, with special focus on the negative feedback by which Ca$^{2+}$ regulates its influx by increasing the closing rate of the channel. This coupling between the channel and the Ca$^{2+}$ makes the channel evolution strongly non-Markovian. We develop a path-integral formulation of the two-state (open/close) process for a single channel, based on the temporal statistics of its state-flips. The feedback effect is built into the model in a systematic way in the form of a weak perturbation. Analytic results are obtained for the open probability of the channel as well as the auto-correlation and response functions (both for the channel variable $S$ and the Ca${^2}$ concentration). Monte Carlo simulations are performed which support the analytical predictions in the weak feedback limit and provide results beyond linear pertubation theory.