Oliphant was born in Adelaide in 1901 and attended the local university before going to Cambridge University in 1927, where he worked on nuclear physics experiments with Ernest Rutherford. In 1937 he was elected a fellow of the Royal Society and moved to Birmingham University to be head of department.

During his time at Birmingham Oliphant was a key figure in many of the scientific developments during the second world war. In 1940 two of his team, John Randall and Harry Boot, invented a new type of magnetron that was crucial in the development of radar. Oliphant also played a key role in getting the UK government to realize the significance of the Frisch-Peierls memorandum, which explained that only a relatively small amount of uranium was needed to make an atomic bomb. And with Ernest Lawrence, he played a similar role in the US. Oliphant actually worked at Los Alamos from 1943.

Oliphant was horrified by the bombing of Nagasaki and Hiroshima and later became one of the founders of the Pugwash peace movement. In 1950 he returned to Australia to help set up the Australian National University in Canberra as director of its research school of physical sciences. Although ANU went on to become Australia's foremost research university, Oliphant was unable to fulfil his dream of building the world's most powerful accelerator in Canberra. His machine was rather cruelly dubbed a "White Oliphant".

Oliphant remained associated with the ANU after his retirement in 1963, continuing to campaign for nuclear power and against nuclear weapons, and serving as governor of South Australia from 1971 to 1976.