Tani was born 22 December 1882 in Okayama Prefecture. He graduated from the 15th class of the Imperial Japanese Army Academy in 1903 and from the 24th class of the Army War College, where he became an instructor in 1924. The College used his texts on strategy and tactics as required readings.
He saw service during the Russo-Japanese War and during the First World War, as official observer for the Japanese government in Great Britain.
From 1935 to 1937, Tani was commanding officer of the 6th Division (Imperial Japanese Army), which was assigned to the China Expeditionary Army in December 1937 under the overall command of General Matsui Iwane. The 6th Division fought in North China during the Peiking – Hankow Railway Operation. Shipped south with the Japanese 10th Army, it took part in the end of the Battle of Shanghai, and the Battle of Nanking.
His troops took Nanking on 13 December 1937. The Chinese army had evacuated the city just before it was taken. The ensuing occupation was therefore that of a defenceless city. The Japanese troops nevertheless carried out unspeakable atrocities: massacre, rape, pillaging and destruction were routinely committed. During a six to seven week period, more than 100’000 civilians were killed and thousands of women raped. Against this backdrop, Matsui marched triumphantly into Nanking on 17 December 1937 and remained there for several days.
He then served as Commander in Chief of the Central Defence Army before retiring. For the Second World War, he was recalled from retirement to the command of the IJA 59th Army and Chugoku Army District.
After the end of War, Tani was extradited to the Chinese government to stand trial for war crimes at the Nanjing War Crimes Tribunal.