I like to think I understand advanced metrics fairly well, but I still think advanced defensive metrics tend to be pretty much a big crock of ****. I mean, the real problem remains that defenders get few enough "important" chances that the sample sizes are small. By important chances, I mean chances that don't fit into one of these 2 categories:
1) 90% of Major League players make the play 90% of the time or better
2) nobody makes the play ever
Realistically, how many marginal chances does a defender get in a full season? Somewhere between 20 and 100 depending on his position? So a couple of uncharacteristically good or poor plays can swing your whole season far off of equilibrium. It doesn't mean that advanced defensive metrics can't tell you useful things. In a sense, they are the *most* precise metrics in baseball. Any analysis of what a hitter does is, to an extent, limited by the fact that we are basically a lot less good at numerically estimating how hard a pitch is to hit than we are at estimating how hard a hit is to field. Knowing the velocity and trajectory of a batted baseball is 100% of the relevant information. Hitting also has a lot to do with release point, delivery, etc. which are hard to quantitatively analyze. So in a sense, we can very effectively say how many balls a fielder got to and how often fielders tend to get to balls with similar hit characteristics.
The problem is, as we keep coming back to, that the samples of such balls are so small it takes a long time for guys to amass enough borderline chances to get a good read on how they're performing if they aren't ridiculously good or ridiculously bad. And not many Major League players are either. Watching the Orioles these days I do get to see 2 though... Machado is a ridiculously good 3B, and Mark Trumbo is a hilarious RF.