Ive experimented with I2c EEPROMs and arduinos and DACing the audio. It worked, it was just slow. Have yet to try with teensy to see if the bottle neck was the arudino, which i suspect it was. It coulda been the serial speed wasnt high enough to retrieve and process across the pwm. I do know it worked however because recorded the output, and sped up the result and it sounded perfect. So its just a matter of playing back at the right sample rate.
Here is the v1 experiment if you want to see:
If you just want a solution that works rather than an experiment that might not, get the wav trigger. You might find it a lil cheaper in the long run.
If you rather risk not having a project that works and would rather experiment, go for it, thats how new awesome things get built.
Just keep in mind polyphony is a thing with the wav trigger, and what you build may not have that support.