The ability to clear NVMe warnings is coming soon (should be there within a week or so). NVMe health is very different from SMART and it's implemented differently.
If this was a spinning drive, IMO (I say omit the H -- xkcd/buzzfeed), yeah for sure RMA. That's an error returned by the the drive when all of its internal error recovery mechanisms have failed to do their job. In other words, if you were reading data at the time, you would experience corruptions, or if that data was a part of the OS that was being read, that could have been a system crash.
But ultimately it's your decision. We're just telling you that your drive has returned 12 errors in the past due to some internal failure. What would you do?
Also, I think what would be useful to know here is, how many 960 EVO 500GB have we seen, and how many of those have errors > 0? That's something that I'd like to implement as well, which I think would be a good metric to help you decide whether your errors are out of the ordinary.