everything causes cancer. I wouldnt worry about getting a few whiffs of smoke causing you to get cancer when some people smoke packs a day and dont get cancer.
Im sure there are plenty of things in your home that can cause cancer. Cleaning products, processed foods, anything.
maybe you should try asking them to move somewhere else instead of sitting in your apartment glaring out the window at them.