Answering “What if?” questions is important in many domains. For example, would a patient’s disease progression slow down if I were to give them a dose of drug A? Ideally, we answer our question using an experiment, but this is not always possible (e.g., it may be unethical). As an alternative, we can use non-experimental data to learn models that make counterfactual predictions of what we would observe had we run an experiment. In this paper, we propose a model to make counterfactual predictions about how continuous-time trajectories (time series) respond to sequences of actions taken in continuous-time. We develop our model within the potential outcomes framework of Neyman and Rubin. One challenge is that the assumptions commonly made to learn potential outcome (counterfactual) models from observational data are not applicable in continuous-time as-is. We therefore propose a model using marked point processes and Gaussian processes, and develop alternative assumptions that allow us to learn counterfactual models from continuous-time observational data. We evaluate our approach on two tasks from health care: disease trajectory prediction and personalized treatment planning.