The only thing I'd add to Judge Kleinfeld's opinion in this case are two brief comments. First, the case seems totally easy to me, and the opinion's only seven pages. How'd this one take almost a full year after oral argument to write? Second, I'm sure that the taxpayer is right that the IRS has abated many, many penalties notwithstanding the mandatory language of the statute. The opinion responds by saying "How do we know that for sure?" and by then distinguishing a couple of cases. But I think the more direct answer is that merely because the executive branch, as it executes the laws, may have discretion to abate mandatory penalties doesn't mean that the judiciary has similar discretion. Executives can plea bargain and grant clemency. Judges can't.
But the basic holding of the case seems totally right. "Shall" means "shall".