With the outbreak of hostilities, Williams travelled to Prestonburg in early 1861 and was commissioned colonel of the 5th Kentucky Infantry. He served initially in the Eastern Theater, initially under Humphrey Marshall in southwestern Virginia. He participated in Marshall's ill-fated invasion of eastern Kentucky in 1862; and fought for Marshall at the victory at the Battle of Princeton Court House. Marshall resigned soon thereafter and was sent to eastern Tennessee. Williams' brigade then spent a short period of time fighting for William W. Loring in the Kanawha Valley Campaign of 1862, skirmishing at Fayetteville and reaching Charleston. Loring was then transferred to the Mississippi, replaced by John Echols. He was promoted to brigadier general in late 1862 and assigned temporary command of the Department of Southwestern Virginia, replacing Echols. He was replaced by General Samuel Jones.
He organized a brigade of cavalry and helped resist Ambrose Burnside's invasion of eastern Tennessee in the autumn of 1863, participating in the Battle of Blue Springs. He resigned that command and transferred to Georgia, assuming command of the Kentucky regiments in the cavalry of Joseph Wheeler in June 1864. He received a formal resolution of thanks from the Second Confederate Congress in the fall of 1864 for his actions at the Battle of Saltville. He surrendered in 1865.