Cassius yes, I agree. In regards to the "others", this varies widely between individuals. I have an unusual degree of feeling based empathy, to the point that observing violence causes my body to hurt at the site of another person's injury. So I actually get pain even at the pain of others who are strangers or enemies. I refrain from causing harm because of that.
It's not completely symmetric on the vicarious pleasure end. I do get strong pleasure at knowledge or witnessing pleasure of strangers, very close to as strong as for those I love but not quite. However, I get no pleasure from the pleasure of those I dislike, and the reason is partly that I dislike them due to what they get pleasure from. If they get pleasure from cruelty to others, then their pleasures are directly counter to mine, and I feel no joy when they get their way.