Silver was a Freedom Rider, arrested and incarcerated for 40 days in Mississippi, and she was also among those on the San Francisco Board of Supervisors allegedly targeted by Supervisor Dan White in the Moscone–Milk assassinations, but was saved because she was not in her office at the time of the murders.

Silver served with the California Rural Legal Assistance program and on the faculties of Golden Gate University, Lone Mountain College (now the University of San Francisco), and San Francisco State University. She also founded San Francisco's Chinese American International School in 1982, the first and most modeled Mandarin Chinese immersion program in the United States. In the summer of 2002 she traveled to Afghanistan to explore ways that American citizens could extend a hand of friendship to the Afghan people, and she has founded or co-founded three organizations dedicated in different ways to supporting and promoting education in Afghanistan, particularly of women and girls.

She is the author of the book, Freedom Rider Diary: Smuggled Notes from Parchman Prison, and in her book Self Defense Handgun Ownership and the Independence of Women in a Violent Sexist Society (co-edited with Don B. Kates Jr.), she argued that, by carrying guns, women advance the cause of women's rights: "For men know that throughout all the prior ages of history the bottom line in male-female relationships has always been woman's need for male protection. Women could not live alone for fear of predation by males. So they lived with a male protector and accepted his dictation of their role, either as a condition of receiving his protection, or because he would impose it upon them by physical force, or both. Access to firearms gives women, for the first time in history, the capacity to live independently and apart from men in safety and freedom."