That was an instruction Doby Jr. took to heart when he joined the Duke University baseball team in the 1970s. The head coach of that team was Hall of Famer Enos Slaughter -- a man whose name is, rightly or wrongly, synonymous with the racism that pervaded the game at the time Jackie Robinson and Doby broke the National League and American League color barriers.
It has been alleged that in 1947 Slaughter tried to convince his Cardinals teammates to strike in protest of Robinson's admittance to the Major Leagues. He has also been accused of intentionally spiking Robinson on a play at first base that same season.
Doby Jr. knew these stories when he arrived on Duke's campus, but his father's words inspired him to go into the experience with an open mind.
"When I got to college, [Slaughter] treated me well," Doby Jr. said. "I have good memories of him to this day. That was one of the biggest things my father taught me was to try to respect people."
When Doby died in 2003, the legacy he left behind was that of being the first black man to play in the AL. But his legacy also includes the values he passed on to his children -- Larry Jr., Leslie, Christina, Susan and Kimberly.
"He was a good father, a good husband and a good man," Doby Jr. said.
Doby Jr.'s baseball career didn't pan out nearly as well as his father's. He played three seasons as an outfielder in the farm system of the White Sox in the late '70s and early '80s, but the game passed him by.
"I could run, and I could hit," he said. "It ended up that I couldn't hit the breaking ball, and that's what ended up sending me home. I had a lot of fun and great memories from that. I played against and with a bunch of guys who made it to the big leagues. Unfortunately, it was a marriage I was divorced from. I didn't get to say when it ended."
Soon after it ended, however, a very happy marriage began.
In 1982, Doby became a union stage hand, working at Madison Square Garden in New York City, the Meadowlands Arena in his native New Jersey and Nassau Veterans Memorial Coliseum on Long Island.
The first show Doby Jr. worked at was a Billy Joel concert, and Joel's production manager, Bob Thrasher, ended up asking Doby Jr. if he would consider going on the road with the "Piano Man."
Doby Jr. accepted, and he's been touring as part of Joel's crew ever since.
"It's been a great experience, because of the way you get treated," he said of the job. "When you're getting treated well by somebody and the guy you work for is a normal guy and a down-to-earth guy and a big baseball fan, it makes you want to do a good job. I've gotten to go around the world and make some money and be associated with somebody I'm proud to be associated with."
But the association that has truly shaped the 48-year-old Doby Jr.'s life is the one he has with his father and namesake.
"My father never talked about what he did in the game," Doby Jr. said. "But when he did talk about that stuff, your ears perked up and you had to believe what he said."