A small group which included Brian Fitzpatrick met in Melbourne in December 1935 and decided to establish an organisation dedicated to the preservation of civil liberties. Most of the group had been members of the Book Censorship Abolition League set up in 1934 by W. Macmahon Ball "to protect the rights of readers and booksellers" after the extensive censorship of political literature during the previous two years. In May 1936 the Council for Civil Liberties was formed at a public meeting. At the annual meeting in 1939 the Council became the Australian Council for Civil Liberties and Brian Fitzpatrick was elected general secretary, a position he held until his death.

The aim of the Council as set out in its 1936 constitution was to assist in the maintenance of the rights of citizens, " against infringement by executive or judicial authority contrary to due process of law, or by tendency of governmental or other agencies to use their powers at the expense of the liberties which citizens in this country have enjoyed." The Council defended individuals and groups irrespective of their political affiliations when it considered "they had been treated unjustly or stood in peril of injustice" and "applied itself to informing the public of facts and implications of 'undemocratic' legislation."

In April 1965 Fitzpatrick wrote of the Council, "before, during and for a few years after the War it operated chiefly from Melbourne but with advisory committees in various States. It remains in existence, but – a voluntary organisation now as at all times – of late years has not functioned as a continuing body conducting regular meetings, etc. Our latest national campaign was conducted towards the end of 1960, when clauses of the amending Crime Act (Commonwealth) of that year were vigorously contested. Nowadays we handle individual cases referred to us, and continue as a loose organisation of (chiefly) lawyers and parliamentarians".