Mass drivers are great on remote comet mines; I don't want to be running a freighter 20 billion km round trip every couple years if I can just do a one and done with a mass driver. It's debatable from an efficiency standpoint I suspect, but worth it from a sanity perspective.
It they are travelling towards you then you could just wait 10 years when the comet is closer and collect the minerals then...
...it depends on how important the minerals from that particular comet is.
To be honest I don't tend to mine comet that are too far away, I simply wait until they get closer to mine them then.
But of course you are right and I also believe I said so above, in some cases it is not really worth it. It will depend on the type of micromanagement you are doing and how good you are at setting up automated routes. I usually take the time to calculate routes with my mineral haulers so they stay as busy as possible all the time, I then adjust them once in a while.
I usually keep my mineral haulers rather small for this purpose at around 5000t cargo holds, it tend to give them old engines too as speed is rarely an issue for them. As I upgrade other freighters with higher speed who transport more time sensitive equipment I reuse the engines for the haulers that carry minerals, components and Infrastructure.