Washington, DC (CBS) - As if taxpayers don't have enough to worry about. Thousands of Americans have been conned out of more than $1 million by crooks posing as IRS agents demanding tax payments, according to the U.S. Treasury.
"This is the largest scam of its kind that we have ever seen," said J. Russell George, the Treasury inspector general for tax administration, who says the agency has received more than 20,000 complaints about the fraud.
The sophisticated phone scam has hit victims in every state, tax officials say. Callers claiming to be from the IRS tell intended victims they owe taxes and must pay using a prepaid debit card or wire transfer. The scammers threaten those who refuse to pay with arrest, deportation or loss of a business or driver's license.
To lend the scam credibility, the crooks often know the last four digits of the taxpayer's Social Security number, and the calls are made with spoofed caller identification software that makes it appear the call is originating from the IRS.
In many cases, taxpayers will get follow-up calls that appear to be from their state motor vehicle agency (if a driver's license was threatened) or the police. The scammers also send follow-up emails that mimic the IRS insignia and even appear to be signed by real IRS officials.
"The increasing number of people receiving these unsolicited calls from individuals who fraudulently claim to represent the IRS is alarming," George said. "Particularly during the tax filing season, we want to make sure that innocent taxpayers are alert to this scam so they are not harmed by these criminals. Do not become a victim."
In reality, if you owe taxes, the IRS will contact you by U.S. mail -- not email. The agency never asks for payment via debit card or wire transfer. It never asks you to provide a credit card number over the phone. And it never requests personal or financial information by e-mail, text or social media.
"If someone unexpectedly calls claiming to be from the IRS and uses threatening language if you don't pay immediately, that is a sign that it is not the IRS calling," George said.
If you get a call from someone claiming to be with the IRS asking for a payment, here's what to do:
* If you owe federal taxes, or think you may owe taxes, hang up and call the IRS directly at 1-800-829-1040.
* If you don't owe taxes, call and report the incident to the Treasury inspector general of tax administration at 1-800-366-4484.
* You can also file a complaint with the Federal Trade Commission at www.FTC.gov. Please add "IRS Telephone Scam" to the comments in your complaint.
* If you get an email that's purportedly from the IRS, do not open any attachments or click on any links in the email. Send it to email@example.com.
Taxpayers should also be aware of other unrelated scams (such as a lottery sweepstakes winner) and solicitations (such as debt relief) that fraudulently claim to be from the IRS. You can read more about identified tax scams at the IRS website, www.irs.gov.
© 2014 CBS Interactive Inc.. All Rights Reserved.