I didn't have a very good experience with directory submission but would recommend you to choose the directory the highest editorial discretion and Domain Authority. ;
Yes, submitting to irrelevant and low quality directories will definitely hurt your site. You can submit your site to authoritative directories like DMOZ, Yahoo, Bing for Business, Yelp, Yellow Pages etc. to gain links.
According to Matt Cutts - "Don’t Try To Build Links Through Article Directories" and "Guest blogging is dead at least for link building or SEO purposes".
Iblis Bane: Highly relevant directories should be fine. ; Don't think social bookmarking has been worth it for a long time. ; Certainly not as a deliberate tactic.
Harry Dance: I would say nearly 90% of directories are a no no. Use them as a last measure, better to focus on things which will generate real visitors from referrals. ;
These sources will always generate some sort of authority, even social media. So in short everything but directories, link building should be more PR these days
Edwin Jonk: From the expert panel in this weeks SEO Questions hangout on air on 00:30:54 into the YouTube video:
The cuetimes listed on this page are functional while the live broadcast continues and also once Youtube has finished processing the public broadcast. Processing time varies between 6-24 hours. During this period the clip is viewable in full but the cuetimes cannot be relied on.