From nicholsonjf at gmail.com Sun Dec 1 03:30:10 2013 From: nicholsonjf at gmail.com (James Nicholson) Date: Sat, 30 Nov 2013 18:30:10 -0800 Subject: [Baypiggies] Big-O talk is up on YouTube Message-ID: Hey Everyone, The video of Jeremy Holman's talk about the Big-O can be viewed here: http://youtu.be/1xCSqMbxd_k Also, please subscribe to our Bay Piggies channel on YouTube! If we get 100 subscribers, Google will allow us to live stream ;) Happy holidays! Cheers, James -------------- next part -------------- An HTML attachment was scrubbed... URL: From tiffany.nguyen at appdynamics.com Tue Dec 3 21:13:57 2013 From: tiffany.nguyen at appdynamics.com (Tiffany Nguyen) Date: Tue, 3 Dec 2013 12:13:57 -0800 Subject: [Baypiggies] AppDynamics Looking for a Python Innovator! Message-ID: *About AppDynamics* AppDynamics is the leader in next generation solutions for managing modern, distributed, and extremely complex applications residing in both the cloud and the data center. Our customers include some of the largest web sites on the planet, who use our products to monitor and troubleshoot applications distributed across thousands of servers. With the paradigm shifts to horizontally distributed applications, service oriented architectures, cloud hosting, and agile deployment it is more complex than ever to manage large-scale applications. The innovative AppDynamics technology greatly simplifies managing these modern application architectures. With over 500 high profile customers and year over year revenue growth above 300%, we are disrupting the multi-billion dollar APM (Application Performance Management) market. Founded in 2008 and lead by proven innovators, AppDynamics has rock solid funding from top tier venture capital firms including: Greylock, Lightspeed, Kleiner Perkins and IVP. Check out our 1.5 minute video: http://www.appdynamics.com/player/whydemo.php. *About You* First and foremost, you have a B.S. Degree in CS / EE (or similar) from a top school as well as have advanced practical experience with Python (not just academic knowledge). You are joyful at the prospect of working in a fast paced and collaborative environment. You have excellent written and verbal communication skills as well as effective interpersonal skills. You also have: - 5 + years of proven experience developing large-scale software applications 5+ years of experience in Java or C++ - Strong analytical and problem solving skills - High and robust coding level and a love for solid engineering - Experience with other programming tools, languages and protocols Big Pluses if you have knowledge of: - J2EE and related technologies a strong plus - Deployment of Python based Web Applications - PHP and/or Node.js *About the Role* This is an opportunity with great rewards for a strong Python Technologist who sees the big picture and delights customers with their work. You will be responsible for building everything from proof-of-concepts and usability prototypes to deployment-quality code. You should be comfortable with daily code submissions, delivering projects in short time frames, multi-tasking, handling interrupts, and interacting position requires hands on development for someone with at least 5 years of experience developing applications. You will also: - Participate in design, implementation and maintenance of complex, multiple product modules/sub-systems - Participate in product design meetings - Perform complex bug verification, release testing and beta support for assigned products - Research problems discovered by QA or product support and develop solutions to the problems - Conduct unit testing and integration testing for functionality and limits - Assist in performing on-site client work on complex systems - Deliver product demos to prove to prospects that we are the best fit solution for their needs - Document development work - Research and understand the marketing requirements for a product, including target environment, performance criteria and competitive issues - Research new technology or development tools to remain informed of current technology *Perks* We know that the award winning culture at AppDynamics is something to brag about, but here are more reasons that make you excited to get out of bed to come in the morning, like: - 100% paid for benefits, including medical, dental, vision, life insurance - Weekly catered breakfast and lunch, and all the snacks, fruits and drinks your heart desires, monthly happy hour events, and weekly massages - Flexible hours (we won?t make you leave your warm bed too early) - Brand new state of the art office in downtown SF, centrally located near BART, Caltrain, Muni, the ferry, and a bike share station, as well as pre-tax commuter benefits - Building offers close lunch destinations, a 24-hour Fitness gym, and weekly Farmers market *Company Highlights* *Breaking News: AppDynamics Nears IPO ! *Forbes ranks AppDynamics as The Best Cloud Computing Company to Work For in 2013! *AppDynamics featured on Bloomberg TV ! *AppDynamics' CEO, Jyoti Bansal on MSNBC ! Thank you! Tiffany *Tiffany Nguyen * | Talent Acquisition Lead *AppDynamics* Monitor your Apps in Production. Now. Watch our Video | Download AppDynamics for Free LinkedIn | Twitter| Facebook | appdynamics.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From vikthirtyfive at gmail.com Thu Dec 5 09:06:06 2013 From: vikthirtyfive at gmail.com (Vikram K) Date: Thu, 5 Dec 2013 03:06:06 -0500 Subject: [Baypiggies] manipulating lists question Message-ID: Any suggestions on what i have to do to go from x to y? >>> x = [['cat','NM123',12],['cat','NM234',12], ['dog', 'NM56',65]] >>> x [['cat', 'NM123', 12], ['cat', 'NM234', 12], ['dog', 'NM56', 65]] >>> y = [] >>> y = [['cat','NM123, NM234', 12], ['dog', 'NM56', 65]] >>> Regards Vik -------------- next part -------------- An HTML attachment was scrubbed... URL: From motoom at xs4all.nl Thu Dec 5 09:53:10 2013 From: motoom at xs4all.nl (Michiel Overtoom) Date: Thu, 5 Dec 2013 09:53:10 +0100 Subject: [Baypiggies] manipulating lists question In-Reply-To: References: Message-ID: On Dec 5, 2013, at 09:06, Vikram K wrote: > Any suggestions on what i have to do to go from x to y? > > >>> x = [['cat','NM123',12],['cat','NM234',12], ['dog', 'NM56',65]] > >>> y = [['cat','NM123, NM234', 12], ['dog', 'NM56', 65]] Are you sure you mean [['cat','NM123, NM234', 12], ['dog', 'NM56', 65]] and not [['cat','NM123', 'NM234', 12], ['dog', 'NM56', 65]] ? > >>> > > Regards > Vik > _______________________________________________ > Baypiggies mailing list > Baypiggies at python.org > To change your subscription options or unsubscribe: > https://mail.python.org/mailman/listinfo/baypiggies -- Veel spullen zijn tegenwoordig waardeloos, maar toch hecht men er veel waarde aan. Mensen hebben vaak teveel spullen. Hoe komt dat en wat doe je ertegen? - een essay van Paul Graham, op http://www.michielovertoom.com/articles/paul-graham-spullen "Een prima telefoon met draaischijf weggooien? Die zou ik nog wel eens nodig kunnen hebben!" From vikthirtyfive at gmail.com Thu Dec 5 10:09:23 2013 From: vikthirtyfive at gmail.com (Vikram K) Date: Thu, 5 Dec 2013 04:09:23 -0500 Subject: [Baypiggies] Fwd: manipulating lists question In-Reply-To: References: Message-ID: yes i am sure. another option could have been to obtain a dictionary like so: >>> x [['cat', 'NM123', 12], ['cat', 'NM234', 12], ['dog', 'NM56', 65]] >>> zee {'dog': ['NM56', 65], 'cat': ['NM123,NM234', 12]} --- The problem is to go from x to zee. On Thu, Dec 5, 2013 at 3:53 AM, Michiel Overtoom wrote: > > On Dec 5, 2013, at 09:06, Vikram K wrote: > > > Any suggestions on what i have to do to go from x to y? > > > > >>> x = [['cat','NM123',12],['cat','NM234',12], ['dog', 'NM56',65]] > > >>> y = [['cat','NM123, NM234', 12], ['dog', 'NM56', 65]] > > Are you sure you mean > > [['cat','NM123, NM234', 12], ['dog', 'NM56', 65]] > > and not > > [['cat','NM123', 'NM234', 12], ['dog', 'NM56', 65]] > > ? > > > > >>> > > > > Regards > > Vik > > _______________________________________________ > > Baypiggies mailing list > > Baypiggies at python.org > > To change your subscription options or unsubscribe: > > https://mail.python.org/mailman/listinfo/baypiggies > > > -- > Veel spullen zijn tegenwoordig waardeloos, maar toch hecht men er veel > waarde aan. Mensen hebben vaak teveel spullen. Hoe komt dat en wat doe je > ertegen? - een essay van Paul Graham, op > http://www.michielovertoom.com/articles/paul-graham-spullen > > "Een prima telefoon met draaischijf weggooien? Die zou ik nog wel eens > nodig kunnen hebben!" > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From motoom at xs4all.nl Thu Dec 5 10:14:58 2013 From: motoom at xs4all.nl (Michiel Overtoom) Date: Thu, 5 Dec 2013 10:14:58 +0100 Subject: [Baypiggies] Fwd: manipulating lists question In-Reply-To: References: Message-ID: On Dec 5, 2013, at 10:09, Vikram K wrote: > yes i am sure In that case, I would use something like this: x = [['cat', 'NM123', 12], ['cat', 'NM234', 12], ['dog', 'NM56', 65]] import collections d = collections.OrderedDict() for key, label, quant in x: if key in d: d[key][0] += ", " + label else: d[key] = [label, quant] y = list(d.iteritems()) print y Greetings, -- "If you don't know, the thing to do is not to get scared, but to learn." - Ayn Rand From michaelglennmoore at gmail.com Thu Dec 5 10:07:35 2013 From: michaelglennmoore at gmail.com (Michael Moore) Date: Thu, 5 Dec 2013 00:07:35 -0900 Subject: [Baypiggies] manipulating lists question In-Reply-To: References: Message-ID: There are many ways. Here is ONE of them >>> def squish(a_list, b_list): res = [] matching = False for idx, i in enumerate(a_list): if i == b_list[idx]: matching = True res.append(i) continue if matching: # note this does not happen unless dissimilar res.append(i) res.append(b_list[idx]) else: res.append(i) return res, matching >>> while len(x): res, matching = squish(x[0], x[1]) if matching: y.append(res) del x[0] del x[1] else: y.append(x[0]) del x[0] if len(x) == 1: y.append(x[0]) x = [] >>> y [['cat', 'NM123', 'NM234', 12], ['cat', 'NM234', 12]] The problem here is ... Definition is fuzzy. Should that happen for every match even if lists with matching multiples even if they are not adjacent (leaving that as an exercise since the problem was not well-defined). Michael Moore On Wed, Dec 4, 2013 at 11:06 PM, Vikram K wrote: > Any suggestions on what i have to do to go from x to y? > > >>> x = [['cat','NM123',12],['cat','NM234',12], ['dog', 'NM56',65]] > >>> x > [['cat', 'NM123', 12], ['cat', 'NM234', 12], ['dog', 'NM56', 65]] > >>> y = [] > >>> y = [['cat','NM123, NM234', 12], ['dog', 'NM56', 65]] > >>> > > Regards > Vik > > _______________________________________________ > Baypiggies mailing list > Baypiggies at python.org > To change your subscription options or unsubscribe: > https://mail.python.org/mailman/listinfo/baypiggies > -------------- next part -------------- An HTML attachment was scrubbed... URL: From motoom at xs4all.nl Thu Dec 5 10:18:13 2013 From: motoom at xs4all.nl (Michiel Overtoom) Date: Thu, 5 Dec 2013 10:18:13 +0100 Subject: [Baypiggies] Fwd: manipulating lists question In-Reply-To: References: Message-ID: <99D5C85E-3796-4056-95B4-928A9C8E85C9@xs4all.nl> On Dec 5, 2013, at 10:09, Vikram K wrote: > another option could have been to obtain a dictionary like so: > {'dog': ['NM56', 65], 'cat': ['NM123,NM234', 12]} Oh, in that case the code can become somewhat simpler: x = [['cat', 'NM123', 12], ['cat', 'NM234', 12], ['dog', 'NM56', 65]] d = {} for key, label, quant in x: if key in d: d[key][0] += ", " + label else: d[key] = [label, quant] print d I agree with Michael that the problem is somewhat underspecified, but it's a starting point. Greetings, -- "If you don't know, the thing to do is not to get scared, but to learn." - Ayn Rand From michaelglennmoore at gmail.com Thu Dec 5 10:29:01 2013 From: michaelglennmoore at gmail.com (Michael Moore) Date: Thu, 5 Dec 2013 00:29:01 -0900 Subject: [Baypiggies] manipulating lists question In-Reply-To: References: Message-ID: Actually there is a hidden flaw in that code...You need to start with the proper x, not with the one processed by an earlier failed attempt and the remove/del haas side effects, better to use slices. >>> x = [['cat','NM123',12],['cat','NM234',12], ['dog', 'NM56',65]] >>> y = [] >>> while len(x): if len(x) == 1: y.append(x[0]) x=[] continue res, matching = squish(x[0], x[1]) if matching: y.append(res) x = x[2:] continue else: y.append(x[0]) x = x[1:] continue >>> y [['cat', 'NM123', 'NM234', 12], ['dog', 'NM56', 65]] >>> On Thu, Dec 5, 2013 at 12:07 AM, Michael Moore wrote: > There are many ways. Here is ONE of them > > >>> def squish(a_list, b_list): > res = [] > matching = False > for idx, i in enumerate(a_list): > if i == b_list[idx]: > matching = True > res.append(i) > continue > if matching: # note this does not happen unless dissimilar > res.append(i) > res.append(b_list[idx]) > else: > res.append(i) > return res, matching > > >>> while len(x): > res, matching = squish(x[0], x[1]) > if matching: > y.append(res) > del x[0] > del x[1] > else: > y.append(x[0]) > del x[0] > if len(x) == 1: > y.append(x[0]) > x = [] > > >>> y > [['cat', 'NM123', 'NM234', 12], ['cat', 'NM234', 12]] > > The problem here is ... Definition is fuzzy. Should that happen for > every match even if lists with matching multiples even if they are not > adjacent (leaving that as an exercise since the problem was not > well-defined). > > Michael Moore > > > > On Wed, Dec 4, 2013 at 11:06 PM, Vikram K wrote: > >> Any suggestions on what i have to do to go from x to y? >> >> >>> x = [['cat','NM123',12],['cat','NM234',12], ['dog', 'NM56',65]] >> >>> x >> [['cat', 'NM123', 12], ['cat', 'NM234', 12], ['dog', 'NM56', 65]] >> >>> y = [] >> >>> y = [['cat','NM123, NM234', 12], ['dog', 'NM56', 65]] >> >>> >> >> Regards >> Vik >> >> _______________________________________________ >> Baypiggies mailing list >> Baypiggies at python.org >> To change your subscription options or unsubscribe: >> https://mail.python.org/mailman/listinfo/baypiggies >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vikthirtyfive at gmail.com Thu Dec 5 10:29:29 2013 From: vikthirtyfive at gmail.com (Vikram K) Date: Thu, 5 Dec 2013 04:29:29 -0500 Subject: [Baypiggies] Fwd: manipulating lists question In-Reply-To: <99D5C85E-3796-4056-95B4-928A9C8E85C9@xs4all.nl> References: <99D5C85E-3796-4056-95B4-928A9C8E85C9@xs4all.nl> Message-ID: much simpler. Thanks to Michiel, Michael, Filip, and Monte for their helpful responses. On Thu, Dec 5, 2013 at 4:18 AM, Michiel Overtoom wrote: > > On Dec 5, 2013, at 10:09, Vikram K wrote: > > > another option could have been to obtain a dictionary like so: > > {'dog': ['NM56', 65], 'cat': ['NM123,NM234', 12]} > > Oh, in that case the code can become somewhat simpler: > > x = [['cat', 'NM123', 12], ['cat', 'NM234', 12], ['dog', 'NM56', 65]] > > d = {} > for key, label, quant in x: > if key in d: > d[key][0] += ", " + label > else: > d[key] = [label, quant] > > print d > > > I agree with Michael that the problem is somewhat underspecified, but it's > a starting point. > > Greetings, > > -- > "If you don't know, the thing to do is not to get scared, but to learn." - > Ayn Rand > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From davidoff56 at alluvialsw.com Thu Dec 5 09:57:45 2013 From: davidoff56 at alluvialsw.com (Monte Davidoff) Date: Thu, 05 Dec 2013 00:57:45 -0800 Subject: [Baypiggies] manipulating lists question In-Reply-To: References: Message-ID: <52A04009.4010104@alluvialsw.com> On 12/5/13 12:06 AM, Vikram K wrote: > Any suggestions on what i have to do to go from x to y? > > >>> x > [['cat', 'NM123', 12], ['cat', 'NM234', 12], ['dog', 'NM56', 65]] > >>> y = [['cat','NM123, NM234', 12], ['dog', 'NM56', 65]] How about this: from itertools import groupby from operator import itemgetter key = itemgetter(0, 2) y = [[k[0], ', '.join(h[1] for h in g), k[1]] for k, g in groupby(sorted(x, key=key), key)] Monte -------------- next part -------------- An HTML attachment was scrubbed... URL: From vikthirtyfive at gmail.com Thu Dec 5 11:11:11 2013 From: vikthirtyfive at gmail.com (Vikram K) Date: Thu, 5 Dec 2013 05:11:11 -0500 Subject: [Baypiggies] Fwd: manipulating lists question In-Reply-To: <99D5C85E-3796-4056-95B4-928A9C8E85C9@xs4all.nl> References: <99D5C85E-3796-4056-95B4-928A9C8E85C9@xs4all.nl> Message-ID: i am having some difficulty in applying this to my actual problem although i love the dictionary method. Imagine the following three lists are the first, second and third elements of a larger list: >>> comp[6] ['6558', 'NM_001046.2', 'SLC12A2', '6037226', '2', 'chr5', '127502453', '127502454', 'het-ref', 'snp', 'A', 'T', 'A', '185', '113', '184', '112', 'VQHIGH', 'VQHIGH', '', '', '', '', '259974', '9', '6', '6', '15', '6558:NM_001046.2:SLC12A2:CDS:MISSENSE', '6558:NM_001046.2:SLC12A2:CDS:NO-CHANGE', 'PFAM:PF01490:Aa_trans', '', '', '', '0.99', '2', '0.99', '0.998', '1.01', '1.000', '0.5', '0.46', '0.5', '1', '18', '18', '19', 'ref-identical;onlyA', 'snp', '0.072', '-1', 'SQHIGH'] >>> comp[7] ['1302', 'NM_080679.2', 'COL11A2', '6525172', '2', 'chr6', '33271374', '33271376', 'het-ref', 'del', 'GT', '', 'GT', '542', '542', '458', '458', 'VQHIGH', 'VQHIGH', '', '', '', '', '71150', '34', '106', '106', '140', '1302:NM_080679.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC', '1302:NM_080679.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC;1302:NM_080680.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC;1302:NM_080681.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC;6257:NM_021976.3:RXRB:CDS:NO-CHANGE', '', '', '', '', '0.95', '2', '0.98', '0.998', '0.99', '1.000', '0.46', '0.42', '0.5', '0', '102', '102', '102', 'ref-identical;onlyA', 'del', '0.990', '6', 'SQHIGH'] >>> comp[8] ['1302', 'NM_080680.2', 'COL11A2', '6525172', '2', 'chr6', '33271374', '33271376', 'het-ref', 'del', 'GT', '', 'GT', '542', '542', '458', '458', 'VQHIGH', 'VQHIGH', '', '', '', '', '71150', '34', '106', '106', '140', '1302:NM_080680.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC', '1302:NM_080679.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC;1302:NM_080680.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC;1302:NM_080681.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC;6257:NM_021976.3:RXRB:CDS:NO-CHANGE', '', '', '', '', '0.95', '2', '0.98', '0.998', '0.99', '1.000', '0.46', '0.42', '0.5', '0', '102', '102', '102', 'ref-identical;onlyA', 'del', '0.990', '6', 'SQHIGH'] >>> ------ Can we apply the dictionary method to the problem where the key of the dictionary is the first element of the three smaller lists ('6558','1302', '1302'). The second and third elements of the larger list (starting with '1302') need to be collapsed into a single element, based on their second element ( 'NM_080679.2') and ('NM_080680.2') in a way similar to how we had tackled the toy problem: x = [['cat', 'NM123', 12], ['cat', 'NM234', 12], ['dog', 'NM56', 65]] On Thu, Dec 5, 2013 at 4:18 AM, Michiel Overtoom wrote: > > On Dec 5, 2013, at 10:09, Vikram K wrote: > > > another option could have been to obtain a dictionary like so: > > {'dog': ['NM56', 65], 'cat': ['NM123,NM234', 12]} > > Oh, in that case the code can become somewhat simpler: > > x = [['cat', 'NM123', 12], ['cat', 'NM234', 12], ['dog', 'NM56', 65]] > > d = {} > for key, label, quant in x: > if key in d: > d[key][0] += ", " + label > else: > d[key] = [label, quant] > > print d > > > I agree with Michael that the problem is somewhat underspecified, but it's > a starting point. > > Greetings, > > -- > "If you don't know, the thing to do is not to get scared, but to learn." - > Ayn Rand > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From martin at falatic.com Thu Dec 5 11:32:57 2013 From: martin at falatic.com (Martin Falatic) Date: Thu, 5 Dec 2013 02:32:57 -0800 (PST) Subject: [Baypiggies] Fwd: manipulating lists question In-Reply-To: <63483.24.23.181.240.1386238955.squirrel@martin-wwwss5.ssl.supercp.com > References: <99D5C85E-3796-4056-95B4-928A9C8E85C9@xs4all.nl> <63483.24.23.181.240.1386238955.squirrel@martin-wwwss5.ssl.supercp.com> Message-ID: <63654.24.23.181.240.1386239577.squirrel@martin-wwwss5.ssl.supercp.com> Actually, I'll just throw this out there since it's late and I have to get up early... I didn't post my solution originally because the other solutions were so much more elegant and a skosh more Pythonic too. The list split/join is a little overcomplicated but you can easily meld this with the other solutions to get something a bit more concise. However, a minor tweak to it satisfies this problem immediately, if the complication I asked about isn't a concern (if it is, it's pretty easy to fix it) y = dict() for part in x: if part[0] in y: mylist = y[part[0]][0].split(",") if not part[1] in mylist: mylist.append(part[1]) y[part[0]][0] = ",".join(mylist) else: y[part[0]] = part[1:3] # truncate the rest of the data subset print y I got this, which seems like what you wanted: {'6558': ['NM_001046.2', 'SLC12A2'], '1302': ['NM_080679.2,NM_080680.2', 'COL11A2']} when I tested this against this: x=[\ ['6558', 'NM_001046.2', 'SLC12A2', '6037226', '2', 'chr5', '127502453', '127502454', 'het-ref', 'snp', 'A', 'T', 'A', '185', '113', '184', '112', 'VQHIGH', 'VQHIGH', '', '', '', '', '259974', '9', '6', '6', '15', '6558:NM_001046.2:SLC12A2:CDS:MISSENSE', '6558:NM_001046.2:SLC12A2:CDS:NO-CHANGE', 'PFAM:PF01490:Aa_trans', '', '', '', '0.99', '2', '0.99', '0.998', '1.01', '1.000', '0.5', '0.46', '0.5', '1', '18', '18', '19', 'ref-identical;onlyA', 'snp', '0.072', '-1', 'SQHIGH']\ ,\ ['1302', 'NM_080679.2', 'COL11A2', '6525172', '2', 'chr6', '33271374', '33271376', 'het-ref', 'del', 'GT', '', 'GT', '542', '542', '458', '458', 'VQHIGH', 'VQHIGH', '', '', '', '', '71150', '34', '106', '106', '140', '1302:NM_080679.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC', '1302:NM_080679.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC;1302:NM_080680.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC;1302:NM_080681.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC;6257:NM_021976.3:RXRB:CDS:NO-CHANGE', '', '', '', '', '0.95', '2', '0.98', '0.998', '0.99', '1.000', '0.46', '0.42', '0.5', '0', '102', '102', '102', 'ref-identical;onlyA', 'del', '0.990', '6', 'SQHIGH']\ ,\ ['1302', 'NM_080680.2', 'COL11A2', '6525172', '2', 'chr6', '33271374', '33271376', 'het-ref', 'del', 'GT', '', 'GT', '542', '542', '458', '458', 'VQHIGH', 'VQHIGH', '', '', '', '', '71150', '34', '106', '106', '140', '1302:NM_080680.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC', '1302:NM_080679.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC;1302:NM_080680.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC;1302:NM_080681.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC;6257:NM_021976.3:RXRB:CDS:NO-CHANGE', '', '', '', '', '0.95', '2', '0.98', '0.998', '0.99', '1.000', '0.46', '0.42', '0.5', '0', '102', '102', '102', 'ref-identical;onlyA', 'del', '0.990', '6', 'SQHIGH']\ ] On Thu, December 5, 2013 02:22, Martin Falatic wrote: > Ah, genetics! Intriguing... > > > Do you need anything beyond the third elements of each list? Does the > third element always map 1:1 with the first, or could it vary? If so, what > then? > > To refer to the simplified example, could you have this? > x = [['cat', 'NM123', 12], ['cat', 'NM234', 43], ['dog', 'NM56', 65]] > > If so, what is the expected output? > > > - Marty > > > > > On Thu, December 5, 2013 02:11, Vikram K wrote: > >> i am having some difficulty in applying this to my actual problem >> although i love the dictionary method. Imagine the following three >> lists are the first, second and third elements of a larger list: >> >>>>> comp[6] >> ['6558', 'NM_001046.2', 'SLC12A2', '6037226', '2', 'chr5', '127502453', >> '127502454', 'het-ref', 'snp', 'A', 'T', 'A', '185', '113', '184', >> '112', >> 'VQHIGH', 'VQHIGH', '', '', '', '', '259974', '9', '6', '6', '15', >> '6558:NM_001046.2:SLC12A2:CDS:MISSENSE', >> '6558:NM_001046.2:SLC12A2:CDS:NO-CHANGE', 'PFAM:PF01490:Aa_trans', '', >> '', >> '', '0.99', '2', '0.99', '0.998', '1.01', '1.000', '0.5', '0.46', '0.5', >> '1', '18', '18', '19', 'ref-identical;onlyA', 'snp', '0.072', '-1', >> 'SQHIGH'] >> >> >> >>>>> comp[7] >> ['1302', 'NM_080679.2', 'COL11A2', '6525172', '2', 'chr6', '33271374', >> '33271376', 'het-ref', 'del', 'GT', '', 'GT', '542', '542', '458', >> '458', >> 'VQHIGH', 'VQHIGH', '', '', '', '', '71150', '34', '106', '106', '140', >> '1302:NM_080679.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC', >> '1302:NM_080679.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC;1302:NM_080680.2:COL1 >> 1A >> 2:TSS-UPSTREAM:UNKNOWN-INC;1302:NM_080681.2:COL11A2:TSS-UPSTREAM:UNKNOWN >> - >> INC;6257:NM_021976.3:RXRB:CDS:NO-CHANGE', >> '', '', '', '', '0.95', '2', '0.98', '0.998', '0.99', '1.000', '0.46', >> '0.42', '0.5', '0', '102', '102', '102', 'ref-identical;onlyA', 'del', >> '0.990', '6', 'SQHIGH'] >> >> >> >>>>> comp[8] >> ['1302', 'NM_080680.2', 'COL11A2', '6525172', '2', 'chr6', '33271374', >> '33271376', 'het-ref', 'del', 'GT', '', 'GT', '542', '542', '458', >> '458', >> 'VQHIGH', 'VQHIGH', '', '', '', '', '71150', '34', '106', '106', '140', >> '1302:NM_080680.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC', >> '1302:NM_080679.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC;1302:NM_080680.2:COL1 >> 1A >> 2:TSS-UPSTREAM:UNKNOWN-INC;1302:NM_080681.2:COL11A2:TSS-UPSTREAM:UNKNOWN >> - >> INC;6257:NM_021976.3:RXRB:CDS:NO-CHANGE', >> '', '', '', '', '0.95', '2', '0.98', '0.998', '0.99', '1.000', '0.46', >> '0.42', '0.5', '0', '102', '102', '102', 'ref-identical;onlyA', 'del', >> '0.990', '6', 'SQHIGH'] >> >> >>>>> >> >> ------ >> Can we apply the dictionary method to the problem where the key of the >> dictionary is the first element of the three smaller lists >> ('6558','1302', >> '1302'). The second and third elements of the larger list (starting with >> '1302') need to be collapsed into a single element, based on their >> second element ( 'NM_080679.2') and ('NM_080680.2') in a way similar to >> how we had tackled the toy problem: >> >> x = [['cat', 'NM123', 12], ['cat', 'NM234', 12], ['dog', 'NM56', 65]] >> >> >> >> >> >> >> >> On Thu, Dec 5, 2013 at 4:18 AM, Michiel Overtoom >> wrote: >> >> >> >>> >>> On Dec 5, 2013, at 10:09, Vikram K wrote: >>> >>> >>> >>>> another option could have been to obtain a dictionary like so: >>>> {'dog': >>>> ['NM56', 65], 'cat': ['NM123,NM234', 12]} >>>> >>>> >>> >>> Oh, in that case the code can become somewhat simpler: >>> >>> >>> >>> x = [['cat', 'NM123', 12], ['cat', 'NM234', 12], ['dog', 'NM56', 65]] >>> >>> >>> d = {} for key, label, quant in x: if key in d: d[key][0] += ", " + >>> label else: >>> d[key] = [label, quant] >>> >>> print d >>> >>> >>> I agree with Michael that the problem is somewhat underspecified, but >>> it's a starting point. >>> >>> Greetings, >>> >>> >>> >>> -- >>> "If you don't know, the thing to do is not to get scared, but to >>> learn." - >>> Ayn Rand >>> >>> >>> >>> >>> >>> >> _______________________________________________ >> Baypiggies mailing list >> Baypiggies at python.org >> To change your subscription options or unsubscribe: >> https://mail.python.org/mailman/listinfo/baypiggies >> > > From vikthirtyfive at gmail.com Thu Dec 5 11:33:05 2013 From: vikthirtyfive at gmail.com (Vikram K) Date: Thu, 5 Dec 2013 05:33:05 -0500 Subject: [Baypiggies] Fwd: manipulating lists question In-Reply-To: <63483.24.23.181.240.1386238955.squirrel@martin-wwwss5.ssl.supercp.com> References: <99D5C85E-3796-4056-95B4-928A9C8E85C9@xs4all.nl> <63483.24.23.181.240.1386238955.squirrel@martin-wwwss5.ssl.supercp.com> Message-ID: In the example i have given, the second and third elements of the larger list (comp[7] and comp[8]) have a 1:1 mapping after the second element. So i would like to keep the first element as it is and then collapse or merge the second and third elements (comp[7] and comp[8]) into a single element: >>> comp[6] ['6558', 'NM_001046.2', 'SLC12A2', '6037226', '2', 'chr5', '127502453', '127502454', 'het-ref', 'snp', 'A', 'T', 'A', '185', '113', '184', '112', 'VQHIGH', 'VQHIGH', '', '', '', '', '259974', '9', '6', '6', '15', '6558:NM_001046.2:SLC12A2:CDS:MISSENSE', '6558:NM_001046.2:SLC12A2:CDS:NO-CHANGE', 'PFAM:PF01490:Aa_trans', '', '', '', '0.99', '2', '0.99', '0.998', '1.01', '1.000', '0.5', '0.46', '0.5', '1', '18', '18', '19', 'ref-identical;onlyA', 'snp', '0.072', '-1', 'SQHIGH'] >>> comp[7] ['1302', 'NM_080679.2', 'COL11A2', '6525172', '2', 'chr6', '33271374', '33271376', 'het-ref', 'del', 'GT', '', 'GT', '542', '542', '458', '458', 'VQHIGH', 'VQHIGH', '', '', '', '', '71150', '34', '106', '106', '140', '1302:NM_080679.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC', '1302:NM_080679.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC;1302:NM_080680.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC;1302:NM_080681.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC;6257:NM_021976.3:RXRB:CDS:NO-CHANGE', '', '', '', '', '0.95', '2', '0.98', '0.998', '0.99', '1.000', '0.46', '0.42', '0.5', '0', '102', '102', '102', 'ref-identical;onlyA', 'del', '0.990', '6', 'SQHIGH'] >>> comp[8] ['1302', 'NM_080680.2', 'COL11A2', '6525172', '2', 'chr6', '33271374', '33271376', 'het-ref', 'del', 'GT', '', 'GT', '542', '542', '458', '458', 'VQHIGH', 'VQHIGH', '', '', '', '', '71150', '34', '106', '106', '140', '1302:NM_080680.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC', '1302:NM_080679.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC;1302:NM_080680.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC;1302:NM_080681.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC;6257:NM_021976.3:RXRB:CDS:NO-CHANGE', '', '', '', '', '0.95', '2', '0.98', '0.998', '0.99', '1.000', '0.46', '0.42', '0.5', '0', '102', '102', '102', 'ref-identical;onlyA', 'del', '0.990', '6', 'SQHIGH'] After collapsing comp[7] and comp[8] i get: >>> collapsed = ['1302', 'NM_080679.2,NM_080680.2', 'COL11A2', '6525172', '2', 'chr6', '33271374', '33271376', 'het-ref', 'del', 'GT', '', 'GT', '542', '542', '458', '458', 'VQHIGH', 'VQHIGH', '', '', '', '', '71150', '34', '106', '106', '140', '1302:NM_080680.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC', '1302:NM_080679.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC;1302:NM_080680.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC;1302:NM_080681.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC;6257:NM_021976.3:RXRB:CDS:NO-CHANGE', '', '', '', '', '0.95', '2', '0.98', '0.998', '0.99', '1.000', '0.46', '0.42', '0.5', '0', '102', '102', '102', 'ref-identical;onlyA', 'del', '0.990', '6', 'SQHIGH'] So in my larger list, after the modification, comp[6] is the first element and collapsed the second element. >>> On Thu, Dec 5, 2013 at 5:22 AM, Martin Falatic wrote: > Ah, genetics! Intriguing... > > Do you need anything beyond the third elements of each list? Does the > third element always map 1:1 with the first, or could it vary? If so, what > then? > > To refer to the simplified example, could you have this? > x = [['cat', 'NM123', 12], ['cat', 'NM234', 43], ['dog', 'NM56', 65]] > > If so, what is the expected output? > > - Marty > > > > On Thu, December 5, 2013 02:11, Vikram K wrote: > > i am having some difficulty in applying this to my actual problem > > although i love the dictionary method. Imagine the following three lists > > are the first, second and third elements of a larger list: > > > >>>> comp[6] > > ['6558', 'NM_001046.2', 'SLC12A2', '6037226', '2', 'chr5', '127502453', > > '127502454', 'het-ref', 'snp', 'A', 'T', 'A', '185', '113', '184', '112', > > 'VQHIGH', 'VQHIGH', '', '', '', '', '259974', '9', '6', '6', '15', > > '6558:NM_001046.2:SLC12A2:CDS:MISSENSE', > > '6558:NM_001046.2:SLC12A2:CDS:NO-CHANGE', 'PFAM:PF01490:Aa_trans', '', > '', > > '', '0.99', '2', '0.99', '0.998', '1.01', '1.000', '0.5', '0.46', '0.5', > > '1', '18', '18', '19', 'ref-identical;onlyA', 'snp', '0.072', '-1', > > 'SQHIGH'] > > > > > >>>> comp[7] > > ['1302', 'NM_080679.2', 'COL11A2', '6525172', '2', 'chr6', '33271374', > > '33271376', 'het-ref', 'del', 'GT', '', 'GT', '542', '542', '458', '458', > > 'VQHIGH', 'VQHIGH', '', '', '', '', '71150', '34', '106', '106', '140', > > '1302:NM_080679.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC', > > > '1302:NM_080679.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC;1302:NM_080680.2:COL11A > > 2:TSS-UPSTREAM:UNKNOWN-INC;1302:NM_080681.2:COL11A2:TSS-UPSTREAM:UNKNOWN- > > INC;6257:NM_021976.3:RXRB:CDS:NO-CHANGE', > > '', '', '', '', '0.95', '2', '0.98', '0.998', '0.99', '1.000', '0.46', > > '0.42', '0.5', '0', '102', '102', '102', 'ref-identical;onlyA', 'del', > > '0.990', '6', 'SQHIGH'] > > > > > >>>> comp[8] > > ['1302', 'NM_080680.2', 'COL11A2', '6525172', '2', 'chr6', '33271374', > > '33271376', 'het-ref', 'del', 'GT', '', 'GT', '542', '542', '458', '458', > > 'VQHIGH', 'VQHIGH', '', '', '', '', '71150', '34', '106', '106', '140', > > '1302:NM_080680.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC', > > > '1302:NM_080679.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC;1302:NM_080680.2:COL11A > > 2:TSS-UPSTREAM:UNKNOWN-INC;1302:NM_080681.2:COL11A2:TSS-UPSTREAM:UNKNOWN- > > INC;6257:NM_021976.3:RXRB:CDS:NO-CHANGE', > > '', '', '', '', '0.95', '2', '0.98', '0.998', '0.99', '1.000', '0.46', > > '0.42', '0.5', '0', '102', '102', '102', 'ref-identical;onlyA', 'del', > > '0.990', '6', 'SQHIGH'] > > > >>>> > > > > ------ > > Can we apply the dictionary method to the problem where the key of the > > dictionary is the first element of the three smaller lists > ('6558','1302', > > '1302'). The second and third elements of the larger list (starting with > > '1302') need to be collapsed into a single element, based on their > > second element ( 'NM_080679.2') and ('NM_080680.2') in a way similar to > > how we had tackled the toy problem: > > > > x = [['cat', 'NM123', 12], ['cat', 'NM234', 12], ['dog', 'NM56', 65]] > > > > > > > > > > > > > > > > On Thu, Dec 5, 2013 at 4:18 AM, Michiel Overtoom > > wrote: > > > > > >> > >> On Dec 5, 2013, at 10:09, Vikram K wrote: > >> > >> > >>> another option could have been to obtain a dictionary like so: {'dog': > >>> ['NM56', 65], 'cat': ['NM123,NM234', 12]} > >>> > >> > >> Oh, in that case the code can become somewhat simpler: > >> > >> > >> x = [['cat', 'NM123', 12], ['cat', 'NM234', 12], ['dog', 'NM56', 65]] > >> > >> d = {} for key, label, quant in x: if key in d: d[key][0] += ", " + > label > >> else: > >> d[key] = [label, quant] > >> > >> print d > >> > >> > >> I agree with Michael that the problem is somewhat underspecified, but > >> it's a starting point. > >> > >> Greetings, > >> > >> > >> -- > >> "If you don't know, the thing to do is not to get scared, but to learn." > >> - > >> Ayn Rand > >> > >> > >> > >> > >> > > _______________________________________________ > > Baypiggies mailing list > > Baypiggies at python.org > > To change your subscription options or unsubscribe: > > https://mail.python.org/mailman/listinfo/baypiggies > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From martin at falatic.com Thu Dec 5 11:40:31 2013 From: martin at falatic.com (Martin Falatic) Date: Thu, 5 Dec 2013 02:40:31 -0800 (PST) Subject: [Baypiggies] Fwd: manipulating lists question In-Reply-To: <63654.24.23.181.240.1386239577.squirrel@martin-wwwss5.ssl.supercp.com > References: <99D5C85E-3796-4056-95B4-928A9C8E85C9@xs4all.nl> <63483.24.23.181.240.1386238955.squirrel@martin-wwwss5.ssl.supercp.com> <63654.24.23.181.240.1386239577.squirrel@martin-wwwss5.ssl.supercp.com> Message-ID: <63789.24.23.181.240.1386240031.squirrel@martin-wwwss5.ssl.supercp.com> Actually, I would leave my solution as-is functionality-wise... the reason I split and rejoin the string list is to ensure it doesn't have duplicates. It'd be more efficient to simply carry that data as a list or set than a comma-delimited string until one gets to the final output stage though... then you can format it more precisely on output (sorting might be desired, for example). I'll reply to your other note separately. This is,however, the core of it. - Marty On Thu, December 5, 2013 02:32, Martin Falatic wrote: > Actually, I'll just throw this out there since it's late and I have to > get up early... > > I didn't post my solution originally because the other solutions were so > much more elegant and a skosh more Pythonic too. The list split/join is a > little overcomplicated but you can easily meld this with the other > solutions to get something a bit more concise. However, a minor tweak to > it satisfies this problem immediately, if the complication I asked about > isn't a concern (if it is, it's pretty easy to fix it) > > y = dict() for part in x: if part[0] in y: mylist = y[part[0]][0].split(",") > if not part[1] in mylist: mylist.append(part[1]) y[part[0]][0] = > ",".join(mylist) > else: > y[part[0]] = part[1:3] # truncate the rest of the data subset print y > > I got this, which seems like what you wanted: > {'6558': ['NM_001046.2', 'SLC12A2'], '1302': ['NM_080679.2,NM_080680.2', > 'COL11A2']} > > > when I tested this against this: x=[\ ['6558', 'NM_001046.2', 'SLC12A2', > '6037226', '2', 'chr5', > '127502453', '127502454', 'het-ref', 'snp', 'A', 'T', 'A', '185', > '113', '184', '112', 'VQHIGH', 'VQHIGH', '', '', '', '', '259974', > '9', '6', '6', '15', '6558:NM_001046.2:SLC12A2:CDS:MISSENSE', > '6558:NM_001046.2:SLC12A2:CDS:NO-CHANGE', 'PFAM:PF01490:Aa_trans', '', > '', '', '0.99', '2', '0.99', '0.998', '1.01', '1.000', '0.5', '0.46', > '0.5', '1', '18', '18', '19', 'ref-identical;onlyA', 'snp', '0.072', > '-1', 'SQHIGH']\ > ,\ > ['1302', 'NM_080679.2', 'COL11A2', '6525172', '2', 'chr6', '33271374', > '33271376', 'het-ref', 'del', 'GT', '', 'GT', '542', '542', '458', > '458', 'VQHIGH', 'VQHIGH', '', '', '', '', '71150', '34', '106', > '106', '140', '1302:NM_080679.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC', > '1302:NM_080679.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC;1302:NM_080680.2:COL11A > 2:TSS-UPSTREAM:UNKNOWN-INC;1302:NM_080681.2:COL11A2:TSS-UPSTREAM:UNKNOWN- > INC;6257:NM_021976.3:RXRB:CDS:NO-CHANGE', > '', '', '', '', '0.95', '2', '0.98', '0.998', '0.99', '1.000', '0.46', > '0.42', '0.5', '0', '102', '102', '102', 'ref-identical;onlyA', 'del', > '0.990', '6', 'SQHIGH']\ > ,\ > ['1302', 'NM_080680.2', 'COL11A2', '6525172', '2', 'chr6', '33271374', > '33271376', 'het-ref', 'del', 'GT', '', 'GT', '542', '542', '458', > '458', 'VQHIGH', 'VQHIGH', '', '', '', '', '71150', '34', '106', > '106', '140', '1302:NM_080680.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC', > '1302:NM_080679.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC;1302:NM_080680.2:COL11A > 2:TSS-UPSTREAM:UNKNOWN-INC;1302:NM_080681.2:COL11A2:TSS-UPSTREAM:UNKNOWN- > INC;6257:NM_021976.3:RXRB:CDS:NO-CHANGE', > '', '', '', '', '0.95', '2', '0.98', '0.998', '0.99', '1.000', '0.46', > '0.42', '0.5', '0', '102', '102', '102', 'ref-identical;onlyA', 'del', > '0.990', '6', 'SQHIGH']\ > ] > > > On Thu, December 5, 2013 02:22, Martin Falatic wrote: > >> Ah, genetics! Intriguing... >> >> >> >> Do you need anything beyond the third elements of each list? Does the >> third element always map 1:1 with the first, or could it vary? If so, >> what then? >> >> To refer to the simplified example, could you have this? >> x = [['cat', 'NM123', 12], ['cat', 'NM234', 43], ['dog', 'NM56', 65]] >> >> If so, what is the expected output? >> >> >> >> - Marty >> >> >> >> >> >> On Thu, December 5, 2013 02:11, Vikram K wrote: >> >> >>> i am having some difficulty in applying this to my actual problem >>> although i love the dictionary method. Imagine the following three >>> lists are the first, second and third elements of a larger list: >>> >>>>>> comp[6] >>> ['6558', 'NM_001046.2', 'SLC12A2', '6037226', '2', 'chr5', >>> '127502453', >>> '127502454', 'het-ref', 'snp', 'A', 'T', 'A', '185', '113', '184', >>> '112', >>> 'VQHIGH', 'VQHIGH', '', '', '', '', '259974', '9', '6', '6', '15', >>> '6558:NM_001046.2:SLC12A2:CDS:MISSENSE', >>> '6558:NM_001046.2:SLC12A2:CDS:NO-CHANGE', 'PFAM:PF01490:Aa_trans', '', >>> '', >>> '', '0.99', '2', '0.99', '0.998', '1.01', '1.000', '0.5', '0.46', >>> '0.5', >>> '1', '18', '18', '19', 'ref-identical;onlyA', 'snp', '0.072', '-1', >>> 'SQHIGH'] >>> >>> >>> >>> >>>>>> comp[7] >>> ['1302', 'NM_080679.2', 'COL11A2', '6525172', '2', 'chr6', >>> '33271374', >>> '33271376', 'het-ref', 'del', 'GT', '', 'GT', '542', '542', '458', >>> '458', >>> 'VQHIGH', 'VQHIGH', '', '', '', '', '71150', '34', '106', '106', >>> '140', >>> '1302:NM_080679.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC', >>> '1302:NM_080679.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC;1302:NM_080680.2:CO >>> L1 >>> 1A >>> 2:TSS-UPSTREAM:UNKNOWN-INC;1302:NM_080681.2:COL11A2:TSS-UPSTREAM:UNKNO >>> WN >>> - >>> INC;6257:NM_021976.3:RXRB:CDS:NO-CHANGE', >>> '', '', '', '', '0.95', '2', '0.98', '0.998', '0.99', '1.000', '0.46', >>> '0.42', '0.5', '0', '102', '102', '102', 'ref-identical;onlyA', >>> 'del', >>> '0.990', '6', 'SQHIGH'] >>> >>> >>> >>> >>>>>> comp[8] >>> ['1302', 'NM_080680.2', 'COL11A2', '6525172', '2', 'chr6', >>> '33271374', >>> '33271376', 'het-ref', 'del', 'GT', '', 'GT', '542', '542', '458', >>> '458', >>> 'VQHIGH', 'VQHIGH', '', '', '', '', '71150', '34', '106', '106', >>> '140', >>> '1302:NM_080680.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC', >>> '1302:NM_080679.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC;1302:NM_080680.2:CO >>> L1 >>> 1A >>> 2:TSS-UPSTREAM:UNKNOWN-INC;1302:NM_080681.2:COL11A2:TSS-UPSTREAM:UNKNO >>> WN >>> - >>> INC;6257:NM_021976.3:RXRB:CDS:NO-CHANGE', >>> '', '', '', '', '0.95', '2', '0.98', '0.998', '0.99', '1.000', '0.46', >>> '0.42', '0.5', '0', '102', '102', '102', 'ref-identical;onlyA', >>> 'del', >>> '0.990', '6', 'SQHIGH'] >>> >>> >>> >>>>>> >>> >>> ------ >>> Can we apply the dictionary method to the problem where the key of the >>> dictionary is the first element of the three smaller lists >>> ('6558','1302', >>> '1302'). The second and third elements of the larger list (starting >>> with '1302') need to be collapsed into a single element, based on >>> their second element ( 'NM_080679.2') and ('NM_080680.2') in a way >>> similar to how we had tackled the toy problem: >>> >>> x = [['cat', 'NM123', 12], ['cat', 'NM234', 12], ['dog', 'NM56', 65]] >>> >>> >>> >>> >>> >>> >>> >>> >>> On Thu, Dec 5, 2013 at 4:18 AM, Michiel Overtoom >>> wrote: >>> >>> >>> >>> >>>> >>>> On Dec 5, 2013, at 10:09, Vikram K wrote: >>>> >>>> >>>> >>>> >>>>> another option could have been to obtain a dictionary like so: >>>>> {'dog': >>>>> ['NM56', 65], 'cat': ['NM123,NM234', 12]} >>>>> >>>>> >>>>> >>>> >>>> Oh, in that case the code can become somewhat simpler: >>>> >>>> >>>> >>>> >>>> x = [['cat', 'NM123', 12], ['cat', 'NM234', 12], ['dog', 'NM56', >>>> 65]] >>>> >>>> >>>> >>>> d = {} for key, label, quant in x: if key in d: d[key][0] += ", " + >>>> label else: d[key] = [label, quant] >>>> >>>> print d >>>> >>>> >>>> I agree with Michael that the problem is somewhat underspecified, >>>> but it's a starting point. >>>> >>>> Greetings, >>>> >>>> >>>> >>>> >>>> -- >>>> "If you don't know, the thing to do is not to get scared, but to >>>> learn." - Ayn Rand >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>> _______________________________________________ >>> Baypiggies mailing list >>> Baypiggies at python.org >>> To change your subscription options or unsubscribe: >>> https://mail.python.org/mailman/listinfo/baypiggies >>> >>> >> >> > > From martin at falatic.com Thu Dec 5 11:22:35 2013 From: martin at falatic.com (Martin Falatic) Date: Thu, 5 Dec 2013 02:22:35 -0800 (PST) Subject: [Baypiggies] Fwd: manipulating lists question In-Reply-To: References: <99D5C85E-3796-4056-95B4-928A9C8E85C9@xs4all.nl> Message-ID: <63483.24.23.181.240.1386238955.squirrel@martin-wwwss5.ssl.supercp.com> Ah, genetics! Intriguing... Do you need anything beyond the third elements of each list? Does the third element always map 1:1 with the first, or could it vary? If so, what then? To refer to the simplified example, could you have this? x = [['cat', 'NM123', 12], ['cat', 'NM234', 43], ['dog', 'NM56', 65]] If so, what is the expected output? - Marty On Thu, December 5, 2013 02:11, Vikram K wrote: > i am having some difficulty in applying this to my actual problem > although i love the dictionary method. Imagine the following three lists > are the first, second and third elements of a larger list: > >>>> comp[6] > ['6558', 'NM_001046.2', 'SLC12A2', '6037226', '2', 'chr5', '127502453', > '127502454', 'het-ref', 'snp', 'A', 'T', 'A', '185', '113', '184', '112', > 'VQHIGH', 'VQHIGH', '', '', '', '', '259974', '9', '6', '6', '15', > '6558:NM_001046.2:SLC12A2:CDS:MISSENSE', > '6558:NM_001046.2:SLC12A2:CDS:NO-CHANGE', 'PFAM:PF01490:Aa_trans', '', '', > '', '0.99', '2', '0.99', '0.998', '1.01', '1.000', '0.5', '0.46', '0.5', > '1', '18', '18', '19', 'ref-identical;onlyA', 'snp', '0.072', '-1', > 'SQHIGH'] > > >>>> comp[7] > ['1302', 'NM_080679.2', 'COL11A2', '6525172', '2', 'chr6', '33271374', > '33271376', 'het-ref', 'del', 'GT', '', 'GT', '542', '542', '458', '458', > 'VQHIGH', 'VQHIGH', '', '', '', '', '71150', '34', '106', '106', '140', > '1302:NM_080679.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC', > '1302:NM_080679.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC;1302:NM_080680.2:COL11A > 2:TSS-UPSTREAM:UNKNOWN-INC;1302:NM_080681.2:COL11A2:TSS-UPSTREAM:UNKNOWN- > INC;6257:NM_021976.3:RXRB:CDS:NO-CHANGE', > '', '', '', '', '0.95', '2', '0.98', '0.998', '0.99', '1.000', '0.46', > '0.42', '0.5', '0', '102', '102', '102', 'ref-identical;onlyA', 'del', > '0.990', '6', 'SQHIGH'] > > >>>> comp[8] > ['1302', 'NM_080680.2', 'COL11A2', '6525172', '2', 'chr6', '33271374', > '33271376', 'het-ref', 'del', 'GT', '', 'GT', '542', '542', '458', '458', > 'VQHIGH', 'VQHIGH', '', '', '', '', '71150', '34', '106', '106', '140', > '1302:NM_080680.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC', > '1302:NM_080679.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC;1302:NM_080680.2:COL11A > 2:TSS-UPSTREAM:UNKNOWN-INC;1302:NM_080681.2:COL11A2:TSS-UPSTREAM:UNKNOWN- > INC;6257:NM_021976.3:RXRB:CDS:NO-CHANGE', > '', '', '', '', '0.95', '2', '0.98', '0.998', '0.99', '1.000', '0.46', > '0.42', '0.5', '0', '102', '102', '102', 'ref-identical;onlyA', 'del', > '0.990', '6', 'SQHIGH'] > >>>> > > ------ > Can we apply the dictionary method to the problem where the key of the > dictionary is the first element of the three smaller lists ('6558','1302', > '1302'). The second and third elements of the larger list (starting with > '1302') need to be collapsed into a single element, based on their > second element ( 'NM_080679.2') and ('NM_080680.2') in a way similar to > how we had tackled the toy problem: > > x = [['cat', 'NM123', 12], ['cat', 'NM234', 12], ['dog', 'NM56', 65]] > > > > > > > > On Thu, Dec 5, 2013 at 4:18 AM, Michiel Overtoom > wrote: > > >> >> On Dec 5, 2013, at 10:09, Vikram K wrote: >> >> >>> another option could have been to obtain a dictionary like so: {'dog': >>> ['NM56', 65], 'cat': ['NM123,NM234', 12]} >>> >> >> Oh, in that case the code can become somewhat simpler: >> >> >> x = [['cat', 'NM123', 12], ['cat', 'NM234', 12], ['dog', 'NM56', 65]] >> >> d = {} for key, label, quant in x: if key in d: d[key][0] += ", " + label >> else: >> d[key] = [label, quant] >> >> print d >> >> >> I agree with Michael that the problem is somewhat underspecified, but >> it's a starting point. >> >> Greetings, >> >> >> -- >> "If you don't know, the thing to do is not to get scared, but to learn." >> - >> Ayn Rand >> >> >> >> >> > _______________________________________________ > Baypiggies mailing list > Baypiggies at python.org > To change your subscription options or unsubscribe: > https://mail.python.org/mailman/listinfo/baypiggies From martin at falatic.com Thu Dec 5 11:51:54 2013 From: martin at falatic.com (Martin Falatic) Date: Thu, 5 Dec 2013 02:51:54 -0800 (PST) Subject: [Baypiggies] Fwd: manipulating lists question In-Reply-To: References: <99D5C85E-3796-4056-95B4-928A9C8E85C9@xs4all.nl> <63483.24.23.181.240.1386238955.squirrel@martin-wwwss5.ssl.supercp.com> Message-ID: <63996.24.23.181.240.1386240714.squirrel@martin-wwwss5.ssl.supercp.com> My solution works for the first three elements as stated, but what you do with the rest of the elements is tricky if they differ for a given key. For 1302 all the fields in the slice [3:] match each other *except* element 28: '1302:NM_080679.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC' '1302:NM_080680.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC' Does this potentially happen with other elements at times? At this point you're faced with either discarding data or mangling data together. The "collapse" just takes the last [3:] slice encountered (for that remainder of data). Is that acceptable? - Marty On Thu, December 5, 2013 02:33, Vikram K wrote: > In the example i have given, the second and third elements of the larger > list (comp[7] and comp[8]) have a 1:1 mapping after the second element. So > i would like to keep the first element as it is and then collapse or > merge the second and third elements (comp[7] and comp[8]) into a single > element: > > >>>> comp[6] > ['6558', 'NM_001046.2', 'SLC12A2', '6037226', '2', 'chr5', '127502453', > '127502454', 'het-ref', 'snp', 'A', 'T', 'A', '185', '113', '184', '112', > 'VQHIGH', 'VQHIGH', '', '', '', '', '259974', '9', '6', '6', '15', > '6558:NM_001046.2:SLC12A2:CDS:MISSENSE', > '6558:NM_001046.2:SLC12A2:CDS:NO-CHANGE', 'PFAM:PF01490:Aa_trans', '', '', > '', '0.99', '2', '0.99', '0.998', '1.01', '1.000', '0.5', '0.46', '0.5', > '1', '18', '18', '19', 'ref-identical;onlyA', 'snp', '0.072', '-1', > 'SQHIGH'] > > >>>> comp[7] > ['1302', 'NM_080679.2', 'COL11A2', '6525172', '2', 'chr6', '33271374', > '33271376', 'het-ref', 'del', 'GT', '', 'GT', '542', '542', '458', '458', > 'VQHIGH', 'VQHIGH', '', '', '', '', '71150', '34', '106', '106', '140', > '1302:NM_080679.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC', > '1302:NM_080679.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC;1302:NM_080680.2:COL11A > 2:TSS-UPSTREAM:UNKNOWN-INC;1302:NM_080681.2:COL11A2:TSS-UPSTREAM:UNKNOWN- > INC;6257:NM_021976.3:RXRB:CDS:NO-CHANGE', > '', '', '', '', '0.95', '2', '0.98', '0.998', '0.99', '1.000', '0.46', > '0.42', '0.5', '0', '102', '102', '102', 'ref-identical;onlyA', 'del', > '0.990', '6', 'SQHIGH'] > > >>>> comp[8] > ['1302', 'NM_080680.2', 'COL11A2', '6525172', '2', 'chr6', '33271374', > '33271376', 'het-ref', 'del', 'GT', '', 'GT', '542', '542', '458', '458', > 'VQHIGH', 'VQHIGH', '', '', '', '', '71150', '34', '106', '106', '140', > '1302:NM_080680.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC', > '1302:NM_080679.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC;1302:NM_080680.2:COL11A > 2:TSS-UPSTREAM:UNKNOWN-INC;1302:NM_080681.2:COL11A2:TSS-UPSTREAM:UNKNOWN- > INC;6257:NM_021976.3:RXRB:CDS:NO-CHANGE', > '', '', '', '', '0.95', '2', '0.98', '0.998', '0.99', '1.000', '0.46', > '0.42', '0.5', '0', '102', '102', '102', 'ref-identical;onlyA', 'del', > '0.990', '6', 'SQHIGH'] > > > After collapsing comp[7] and comp[8] i get: > > >>>> collapsed = ['1302', 'NM_080679.2,NM_080680.2', 'COL11A2', >>>> '6525172', >>>> > '2', 'chr6', '33271374', '33271376', 'het-ref', 'del', 'GT', '', 'GT', > '542', '542', '458', '458', 'VQHIGH', 'VQHIGH', '', '', '', '', '71150', > '34', '106', '106', '140', > '1302:NM_080680.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC', > '1302:NM_080679.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC;1302:NM_080680.2:COL11A > 2:TSS-UPSTREAM:UNKNOWN-INC;1302:NM_080681.2:COL11A2:TSS-UPSTREAM:UNKNOWN- > INC;6257:NM_021976.3:RXRB:CDS:NO-CHANGE', > '', '', '', '', '0.95', '2', '0.98', '0.998', '0.99', '1.000', '0.46', > '0.42', '0.5', '0', '102', '102', '102', 'ref-identical;onlyA', 'del', > '0.990', '6', 'SQHIGH'] > > > So in my larger list, after the modification, comp[6] is the first > element and collapsed the second element. >>>> > > > On Thu, Dec 5, 2013 at 5:22 AM, Martin Falatic > wrote: > > >> Ah, genetics! Intriguing... >> >> >> Do you need anything beyond the third elements of each list? Does the >> third element always map 1:1 with the first, or could it vary? If so, >> what then? >> >> To refer to the simplified example, could you have this? >> x = [['cat', 'NM123', 12], ['cat', 'NM234', 43], ['dog', 'NM56', 65]] >> >> If so, what is the expected output? >> >> >> - Marty >> >> >> >> >> On Thu, December 5, 2013 02:11, Vikram K wrote: >> >>> i am having some difficulty in applying this to my actual problem >>> although i love the dictionary method. Imagine the following three >>> lists are the first, second and third elements of a larger list: >>> >>>>>> comp[6] >>> ['6558', 'NM_001046.2', 'SLC12A2', '6037226', '2', 'chr5', >>> '127502453', >>> '127502454', 'het-ref', 'snp', 'A', 'T', 'A', '185', '113', '184', >>> '112', >>> 'VQHIGH', 'VQHIGH', '', '', '', '', '259974', '9', '6', '6', '15', >>> '6558:NM_001046.2:SLC12A2:CDS:MISSENSE', >>> '6558:NM_001046.2:SLC12A2:CDS:NO-CHANGE', 'PFAM:PF01490:Aa_trans', '', >>> >> '', >> >>> '', '0.99', '2', '0.99', '0.998', '1.01', '1.000', '0.5', '0.46', >>> '0.5', >>> '1', '18', '18', '19', 'ref-identical;onlyA', 'snp', '0.072', '-1', >>> 'SQHIGH'] >>> >>> >>> >>>>>> comp[7] >>> ['1302', 'NM_080679.2', 'COL11A2', '6525172', '2', 'chr6', >>> '33271374', >>> '33271376', 'het-ref', 'del', 'GT', '', 'GT', '542', '542', '458', >>> '458', >>> 'VQHIGH', 'VQHIGH', '', '', '', '', '71150', '34', '106', '106', >>> '140', >>> '1302:NM_080679.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC', >>> >>> >> '1302:NM_080679.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC;1302:NM_080680.2:COL >> 11A >> >>> 2:TSS-UPSTREAM:UNKNOWN-INC;1302:NM_080681.2:COL11A2:TSS-UPSTREAM:UNKN >>> OWN- >>> INC;6257:NM_021976.3:RXRB:CDS:NO-CHANGE', >>> '', '', '', '', '0.95', '2', '0.98', '0.998', '0.99', '1.000', '0.46', >>> '0.42', '0.5', '0', '102', '102', '102', 'ref-identical;onlyA', >>> 'del', >>> '0.990', '6', 'SQHIGH'] >>> >>> >>> >>>>>> comp[8] >>> ['1302', 'NM_080680.2', 'COL11A2', '6525172', '2', 'chr6', >>> '33271374', >>> '33271376', 'het-ref', 'del', 'GT', '', 'GT', '542', '542', '458', >>> '458', >>> 'VQHIGH', 'VQHIGH', '', '', '', '', '71150', '34', '106', '106', >>> '140', >>> '1302:NM_080680.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC', >>> >>> >> '1302:NM_080679.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC;1302:NM_080680.2:COL >> 11A >> >>> 2:TSS-UPSTREAM:UNKNOWN-INC;1302:NM_080681.2:COL11A2:TSS-UPSTREAM:UNKN >>> OWN- >>> INC;6257:NM_021976.3:RXRB:CDS:NO-CHANGE', >>> '', '', '', '', '0.95', '2', '0.98', '0.998', '0.99', '1.000', '0.46', >>> '0.42', '0.5', '0', '102', '102', '102', 'ref-identical;onlyA', >>> 'del', >>> '0.990', '6', 'SQHIGH'] >>> >>> >>>>>> >>> >>> ------ >>> Can we apply the dictionary method to the problem where the key of the >>> dictionary is the first element of the three smaller lists >> ('6558','1302', >> >>> '1302'). The second and third elements of the larger list (starting >>> with '1302') need to be collapsed into a single element, based on >>> their second element ( 'NM_080679.2') and ('NM_080680.2') in a way >>> similar to how we had tackled the toy problem: >>> >>> x = [['cat', 'NM123', 12], ['cat', 'NM234', 12], ['dog', 'NM56', 65]] >>> >>> >>> >>> >>> >>> >>> >>> >>> On Thu, Dec 5, 2013 at 4:18 AM, Michiel Overtoom >>> wrote: >>> >>> >>> >>>> >>>> On Dec 5, 2013, at 10:09, Vikram K wrote: >>>> >>>> >>>> >>>>> another option could have been to obtain a dictionary like so: >>>>> {'dog': >>>>> ['NM56', 65], 'cat': ['NM123,NM234', 12]} >>>>> >>>>> >>>> >>>> Oh, in that case the code can become somewhat simpler: >>>> >>>> >>>> >>>> x = [['cat', 'NM123', 12], ['cat', 'NM234', 12], ['dog', 'NM56', >>>> 65]] >>>> >>>> >>>> d = {} for key, label, quant in x: if key in d: d[key][0] += ", " + >>>> >> label >>>> else: >>>> d[key] = [label, quant] >>>> >>>> print d >>>> >>>> >>>> I agree with Michael that the problem is somewhat underspecified, >>>> but it's a starting point. >>>> >>>> Greetings, >>>> >>>> >>>> >>>> -- >>>> "If you don't know, the thing to do is not to get scared, but to >>>> learn." - >>>> Ayn Rand >>>> >>>> >>>> >>>> >>>> >>>> >>> _______________________________________________ >>> Baypiggies mailing list >>> Baypiggies at python.org >>> To change your subscription options or unsubscribe: >>> https://mail.python.org/mailman/listinfo/baypiggies >>> >> >> >> > From vikthirtyfive at gmail.com Thu Dec 5 12:06:05 2013 From: vikthirtyfive at gmail.com (Vikram K) Date: Thu, 5 Dec 2013 06:06:05 -0500 Subject: [Baypiggies] Fwd: manipulating lists question In-Reply-To: <63996.24.23.181.240.1386240714.squirrel@martin-wwwss5.ssl.supercp.com> References: <99D5C85E-3796-4056-95B4-928A9C8E85C9@xs4all.nl> <63483.24.23.181.240.1386238955.squirrel@martin-wwwss5.ssl.supercp.com> <63996.24.23.181.240.1386240714.squirrel@martin-wwwss5.ssl.supercp.com> Message-ID: Good catch. All the other elements remain the same except this one. Element 28 needs to be changed (in the merged/collapsed list) so that when we fuse or merge two elements of the larger list into one then Element 28 of the new element is (just combine whatever is present in element 28 in both the lists keeping a ';' as delimiter): '1302:NM_080680.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC'; ' 1302:NM_080679.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC' On Thu, Dec 5, 2013 at 5:51 AM, Martin Falatic wrote: > My solution works for the first three elements as stated, but what you do > with the rest of the elements is tricky if they differ for a given key. > > For 1302 all the fields in the slice [3:] match each other *except* > element 28: > '1302:NM_080679.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC' > '1302:NM_080680.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC' > > Does this potentially happen with other elements at times? At this point > you're faced with either discarding data or mangling data together. The > "collapse" just takes the last [3:] slice encountered (for that remainder > of data). Is that acceptable? > > - Marty > > > On Thu, December 5, 2013 02:33, Vikram K wrote: > > In the example i have given, the second and third elements of the larger > > list (comp[7] and comp[8]) have a 1:1 mapping after the second element. > So > > i would like to keep the first element as it is and then collapse or > > merge the second and third elements (comp[7] and comp[8]) into a single > > element: > > > > > >>>> comp[6] > > ['6558', 'NM_001046.2', 'SLC12A2', '6037226', '2', 'chr5', '127502453', > > '127502454', 'het-ref', 'snp', 'A', 'T', 'A', '185', '113', '184', '112', > > 'VQHIGH', 'VQHIGH', '', '', '', '', '259974', '9', '6', '6', '15', > > '6558:NM_001046.2:SLC12A2:CDS:MISSENSE', > > '6558:NM_001046.2:SLC12A2:CDS:NO-CHANGE', 'PFAM:PF01490:Aa_trans', '', > '', > > '', '0.99', '2', '0.99', '0.998', '1.01', '1.000', '0.5', '0.46', '0.5', > > '1', '18', '18', '19', 'ref-identical;onlyA', 'snp', '0.072', '-1', > > 'SQHIGH'] > > > > > >>>> comp[7] > > ['1302', 'NM_080679.2', 'COL11A2', '6525172', '2', 'chr6', '33271374', > > '33271376', 'het-ref', 'del', 'GT', '', 'GT', '542', '542', '458', '458', > > 'VQHIGH', 'VQHIGH', '', '', '', '', '71150', '34', '106', '106', '140', > > '1302:NM_080679.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC', > > > '1302:NM_080679.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC;1302:NM_080680.2:COL11A > > 2:TSS-UPSTREAM:UNKNOWN-INC;1302:NM_080681.2:COL11A2:TSS-UPSTREAM:UNKNOWN- > > INC;6257:NM_021976.3:RXRB:CDS:NO-CHANGE', > > '', '', '', '', '0.95', '2', '0.98', '0.998', '0.99', '1.000', '0.46', > > '0.42', '0.5', '0', '102', '102', '102', 'ref-identical;onlyA', 'del', > > '0.990', '6', 'SQHIGH'] > > > > > >>>> comp[8] > > ['1302', 'NM_080680.2', 'COL11A2', '6525172', '2', 'chr6', '33271374', > > '33271376', 'het-ref', 'del', 'GT', '', 'GT', '542', '542', '458', '458', > > 'VQHIGH', 'VQHIGH', '', '', '', '', '71150', '34', '106', '106', '140', > > '1302:NM_080680.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC', > > > '1302:NM_080679.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC;1302:NM_080680.2:COL11A > > 2:TSS-UPSTREAM:UNKNOWN-INC;1302:NM_080681.2:COL11A2:TSS-UPSTREAM:UNKNOWN- > > INC;6257:NM_021976.3:RXRB:CDS:NO-CHANGE', > > '', '', '', '', '0.95', '2', '0.98', '0.998', '0.99', '1.000', '0.46', > > '0.42', '0.5', '0', '102', '102', '102', 'ref-identical;onlyA', 'del', > > '0.990', '6', 'SQHIGH'] > > > > > > After collapsing comp[7] and comp[8] i get: > > > > > >>>> collapsed = ['1302', 'NM_080679.2,NM_080680.2', 'COL11A2', > >>>> '6525172', > >>>> > > '2', 'chr6', '33271374', '33271376', 'het-ref', 'del', 'GT', '', 'GT', > > '542', '542', '458', '458', 'VQHIGH', 'VQHIGH', '', '', '', '', '71150', > > '34', '106', '106', '140', > > '1302:NM_080680.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC', > > > '1302:NM_080679.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC;1302:NM_080680.2:COL11A > > 2:TSS-UPSTREAM:UNKNOWN-INC;1302:NM_080681.2:COL11A2:TSS-UPSTREAM:UNKNOWN- > > INC;6257:NM_021976.3:RXRB:CDS:NO-CHANGE', > > '', '', '', '', '0.95', '2', '0.98', '0.998', '0.99', '1.000', '0.46', > > '0.42', '0.5', '0', '102', '102', '102', 'ref-identical;onlyA', 'del', > > '0.990', '6', 'SQHIGH'] > > > > > > So in my larger list, after the modification, comp[6] is the first > > element and collapsed the second element. > >>>> > > > > > > On Thu, Dec 5, 2013 at 5:22 AM, Martin Falatic > > wrote: > > > > > >> Ah, genetics! Intriguing... > >> > >> > >> Do you need anything beyond the third elements of each list? Does the > >> third element always map 1:1 with the first, or could it vary? If so, > >> what then? > >> > >> To refer to the simplified example, could you have this? > >> x = [['cat', 'NM123', 12], ['cat', 'NM234', 43], ['dog', 'NM56', 65]] > >> > >> If so, what is the expected output? > >> > >> > >> - Marty > >> > >> > >> > >> > >> On Thu, December 5, 2013 02:11, Vikram K wrote: > >> > >>> i am having some difficulty in applying this to my actual problem > >>> although i love the dictionary method. Imagine the following three > >>> lists are the first, second and third elements of a larger list: > >>> > >>>>>> comp[6] > >>> ['6558', 'NM_001046.2', 'SLC12A2', '6037226', '2', 'chr5', > >>> '127502453', > >>> '127502454', 'het-ref', 'snp', 'A', 'T', 'A', '185', '113', '184', > >>> '112', > >>> 'VQHIGH', 'VQHIGH', '', '', '', '', '259974', '9', '6', '6', '15', > >>> '6558:NM_001046.2:SLC12A2:CDS:MISSENSE', > >>> '6558:NM_001046.2:SLC12A2:CDS:NO-CHANGE', 'PFAM:PF01490:Aa_trans', '', > >>> > >> '', > >> > >>> '', '0.99', '2', '0.99', '0.998', '1.01', '1.000', '0.5', '0.46', > >>> '0.5', > >>> '1', '18', '18', '19', 'ref-identical;onlyA', 'snp', '0.072', '-1', > >>> 'SQHIGH'] > >>> > >>> > >>> > >>>>>> comp[7] > >>> ['1302', 'NM_080679.2', 'COL11A2', '6525172', '2', 'chr6', > >>> '33271374', > >>> '33271376', 'het-ref', 'del', 'GT', '', 'GT', '542', '542', '458', > >>> '458', > >>> 'VQHIGH', 'VQHIGH', '', '', '', '', '71150', '34', '106', '106', > >>> '140', > >>> '1302:NM_080679.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC', > >>> > >>> > >> '1302:NM_080679.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC;1302:NM_080680.2:COL > >> 11A > >> > >>> 2:TSS-UPSTREAM:UNKNOWN-INC;1302:NM_080681.2:COL11A2:TSS-UPSTREAM:UNKN > >>> OWN- > >>> INC;6257:NM_021976.3:RXRB:CDS:NO-CHANGE', > >>> '', '', '', '', '0.95', '2', '0.98', '0.998', '0.99', '1.000', '0.46', > >>> '0.42', '0.5', '0', '102', '102', '102', 'ref-identical;onlyA', > >>> 'del', > >>> '0.990', '6', 'SQHIGH'] > >>> > >>> > >>> > >>>>>> comp[8] > >>> ['1302', 'NM_080680.2', 'COL11A2', '6525172', '2', 'chr6', > >>> '33271374', > >>> '33271376', 'het-ref', 'del', 'GT', '', 'GT', '542', '542', '458', > >>> '458', > >>> 'VQHIGH', 'VQHIGH', '', '', '', '', '71150', '34', '106', '106', > >>> '140', > >>> '1302:NM_080680.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC', > >>> > >>> > >> '1302:NM_080679.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC;1302:NM_080680.2:COL > >> 11A > >> > >>> 2:TSS-UPSTREAM:UNKNOWN-INC;1302:NM_080681.2:COL11A2:TSS-UPSTREAM:UNKN > >>> OWN- > >>> INC;6257:NM_021976.3:RXRB:CDS:NO-CHANGE', > >>> '', '', '', '', '0.95', '2', '0.98', '0.998', '0.99', '1.000', '0.46', > >>> '0.42', '0.5', '0', '102', '102', '102', 'ref-identical;onlyA', > >>> 'del', > >>> '0.990', '6', 'SQHIGH'] > >>> > >>> > >>>>>> > >>> > >>> ------ > >>> Can we apply the dictionary method to the problem where the key of the > >>> dictionary is the first element of the three smaller lists > >> ('6558','1302', > >> > >>> '1302'). The second and third elements of the larger list (starting > >>> with '1302') need to be collapsed into a single element, based on > >>> their second element ( 'NM_080679.2') and ('NM_080680.2') in a way > >>> similar to how we had tackled the toy problem: > >>> > >>> x = [['cat', 'NM123', 12], ['cat', 'NM234', 12], ['dog', 'NM56', 65]] > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> On Thu, Dec 5, 2013 at 4:18 AM, Michiel Overtoom > >>> wrote: > >>> > >>> > >>> > >>>> > >>>> On Dec 5, 2013, at 10:09, Vikram K wrote: > >>>> > >>>> > >>>> > >>>>> another option could have been to obtain a dictionary like so: > >>>>> {'dog': > >>>>> ['NM56', 65], 'cat': ['NM123,NM234', 12]} > >>>>> > >>>>> > >>>> > >>>> Oh, in that case the code can become somewhat simpler: > >>>> > >>>> > >>>> > >>>> x = [['cat', 'NM123', 12], ['cat', 'NM234', 12], ['dog', 'NM56', > >>>> 65]] > >>>> > >>>> > >>>> d = {} for key, label, quant in x: if key in d: d[key][0] += ", " + > >>>> > >> label > >>>> else: > >>>> d[key] = [label, quant] > >>>> > >>>> print d > >>>> > >>>> > >>>> I agree with Michael that the problem is somewhat underspecified, > >>>> but it's a starting point. > >>>> > >>>> Greetings, > >>>> > >>>> > >>>> > >>>> -- > >>>> "If you don't know, the thing to do is not to get scared, but to > >>>> learn." - > >>>> Ayn Rand > >>>> > >>>> > >>>> > >>>> > >>>> > >>>> > >>> _______________________________________________ > >>> Baypiggies mailing list > >>> Baypiggies at python.org > >>> To change your subscription options or unsubscribe: > >>> https://mail.python.org/mailman/listinfo/baypiggies > >>> > >> > >> > >> > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From martin at falatic.com Thu Dec 5 12:21:05 2013 From: martin at falatic.com (Martin Falatic) Date: Thu, 5 Dec 2013 03:21:05 -0800 (PST) Subject: [Baypiggies] Fwd: manipulating lists question In-Reply-To: References: <99D5C85E-3796-4056-95B4-928A9C8E85C9@xs4all.nl> <63483.24.23.181.240.1386238955.squirrel@martin-wwwss5.ssl.supercp.com> <63996.24.23.181.240.1386240714.squirrel@martin-wwwss5.ssl.supercp.com> Message-ID: <64669.24.23.181.240.1386242465.squirrel@martin-wwwss5.ssl.supercp.com> One can leave that as an exercise for the reader. :-) I'm not sure why this gets a ';' versus a ',', nor is it clear if these field lists are supposed to be deduped or ordered or what... consider if you have three sets of data for 1302, and two ONLY vary by this field 28. If you go to reconstruct the data set you end up with a somewhat mangled thing. I take it this is an effort to compress the original data set to a more manageable size for output / internal representation, which suggests deduping isn't desirable (but which also suggests that you should simply include every instance for fields 1 (which we already do) and 28, and hope none of the other fields vary). On that note I'll throw this idea out there: given key field 0 as identical for n sets of data, for every subsequent field [1:] if the item is a str or int, consider it duplicated for all n sets. If the item is a list then it much have exactly n elements (in the order the n sets were parsed). That way if another fields is found to vary unexpectedly, it'll simply become a list of n elements (many might be the same). You can always take that and stringify the elements and lists for storage or whatever. The idea is that your internal data representation is a much easier to work with set of lists/strs/ints within dictionary entries. - Marty On Thu, December 5, 2013 03:06, Vikram K wrote: > Good catch. All the other elements remain the same except this one. > Element > 28 needs to be changed (in the merged/collapsed list) so that when we fuse > or merge two elements of the larger list into one then Element 28 of the > new element is (just combine whatever is present in element 28 in both > the lists keeping a ';' as delimiter): > > '1302:NM_080680.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC'; ' > 1302:NM_080679.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC' > > > > > On Thu, Dec 5, 2013 at 5:51 AM, Martin Falatic > wrote: > > >> My solution works for the first three elements as stated, but what you >> do with the rest of the elements is tricky if they differ for a given >> key. >> >> For 1302 all the fields in the slice [3:] match each other *except* >> element 28: '1302:NM_080679.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC' >> '1302:NM_080680.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC' >> >> >> Does this potentially happen with other elements at times? At this >> point you're faced with either discarding data or mangling data >> together. The "collapse" just takes the last [3:] slice encountered (for >> that remainder of data). Is that acceptable? >> >> - Marty >> >> >> >> On Thu, December 5, 2013 02:33, Vikram K wrote: >> >>> In the example i have given, the second and third elements of the >>> larger list (comp[7] and comp[8]) have a 1:1 mapping after the second >>> element. >> So >> >>> i would like to keep the first element as it is and then collapse or >>> merge the second and third elements (comp[7] and comp[8]) into a >>> single element: >>> >>> >>> >>>>>> comp[6] >>> ['6558', 'NM_001046.2', 'SLC12A2', '6037226', '2', 'chr5', >>> '127502453', >>> '127502454', 'het-ref', 'snp', 'A', 'T', 'A', '185', '113', '184', >>> '112', >>> 'VQHIGH', 'VQHIGH', '', '', '', '', '259974', '9', '6', '6', '15', >>> '6558:NM_001046.2:SLC12A2:CDS:MISSENSE', >>> '6558:NM_001046.2:SLC12A2:CDS:NO-CHANGE', 'PFAM:PF01490:Aa_trans', '', >>> >> '', >> >>> '', '0.99', '2', '0.99', '0.998', '1.01', '1.000', '0.5', '0.46', >>> '0.5', >>> '1', '18', '18', '19', 'ref-identical;onlyA', 'snp', '0.072', '-1', >>> 'SQHIGH'] >>> >>> >>> >>>>>> comp[7] >>> ['1302', 'NM_080679.2', 'COL11A2', '6525172', '2', 'chr6', >>> '33271374', >>> '33271376', 'het-ref', 'del', 'GT', '', 'GT', '542', '542', '458', >>> '458', >>> 'VQHIGH', 'VQHIGH', '', '', '', '', '71150', '34', '106', '106', >>> '140', >>> '1302:NM_080679.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC', >>> >>> >> '1302:NM_080679.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC;1302:NM_080680.2:COL >> 11A >> >>> 2:TSS-UPSTREAM:UNKNOWN-INC;1302:NM_080681.2:COL11A2:TSS-UPSTREAM:UNKN >>> OWN- >>> INC;6257:NM_021976.3:RXRB:CDS:NO-CHANGE', >>> '', '', '', '', '0.95', '2', '0.98', '0.998', '0.99', '1.000', '0.46', >>> '0.42', '0.5', '0', '102', '102', '102', 'ref-identical;onlyA', >>> 'del', >>> '0.990', '6', 'SQHIGH'] >>> >>> >>> >>>>>> comp[8] >>> ['1302', 'NM_080680.2', 'COL11A2', '6525172', '2', 'chr6', >>> '33271374', >>> '33271376', 'het-ref', 'del', 'GT', '', 'GT', '542', '542', '458', >>> '458', >>> 'VQHIGH', 'VQHIGH', '', '', '', '', '71150', '34', '106', '106', >>> '140', >>> '1302:NM_080680.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC', >>> >>> >> '1302:NM_080679.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC;1302:NM_080680.2:COL >> 11A >> >>> 2:TSS-UPSTREAM:UNKNOWN-INC;1302:NM_080681.2:COL11A2:TSS-UPSTREAM:UNKN >>> OWN- >>> INC;6257:NM_021976.3:RXRB:CDS:NO-CHANGE', >>> '', '', '', '', '0.95', '2', '0.98', '0.998', '0.99', '1.000', '0.46', >>> '0.42', '0.5', '0', '102', '102', '102', 'ref-identical;onlyA', >>> 'del', >>> '0.990', '6', 'SQHIGH'] >>> >>> >>> >>> After collapsing comp[7] and comp[8] i get: >>> >>> >>> >>>>>> collapsed = ['1302', 'NM_080679.2,NM_080680.2', 'COL11A2', >>>>>> '6525172', >>>>>> >>>>>> >>> '2', 'chr6', '33271374', '33271376', 'het-ref', 'del', 'GT', '', >>> 'GT', >>> '542', '542', '458', '458', 'VQHIGH', 'VQHIGH', '', '', '', '', >>> '71150', >>> '34', '106', '106', '140', >>> '1302:NM_080680.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC', >>> >>> >> '1302:NM_080679.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC;1302:NM_080680.2:COL >> 11A >> >>> 2:TSS-UPSTREAM:UNKNOWN-INC;1302:NM_080681.2:COL11A2:TSS-UPSTREAM:UNKN >>> OWN- >>> INC;6257:NM_021976.3:RXRB:CDS:NO-CHANGE', >>> '', '', '', '', '0.95', '2', '0.98', '0.998', '0.99', '1.000', '0.46', >>> '0.42', '0.5', '0', '102', '102', '102', 'ref-identical;onlyA', >>> 'del', >>> '0.990', '6', 'SQHIGH'] >>> >>> >>> >>> So in my larger list, after the modification, comp[6] is the first >>> element and collapsed the second element. >>>>>> >>> >>> >>> On Thu, Dec 5, 2013 at 5:22 AM, Martin Falatic >>> wrote: >>> >>> >>> >>>> Ah, genetics! Intriguing... >>>> >>>> >>>> >>>> Do you need anything beyond the third elements of each list? Does >>>> the third element always map 1:1 with the first, or could it vary? >>>> If so, >>>> what then? >>>> >>>> To refer to the simplified example, could you have this? >>>> x = [['cat', 'NM123', 12], ['cat', 'NM234', 43], ['dog', 'NM56', >>>> 65]] >>>> >>>> >>>> If so, what is the expected output? >>>> >>>> >>>> >>>> - Marty >>>> >>>> >>>> >>>> >>>> >>>> On Thu, December 5, 2013 02:11, Vikram K wrote: >>>> >>>> >>>>> i am having some difficulty in applying this to my actual problem >>>>> although i love the dictionary method. Imagine the following >>>>> three lists are the first, second and third elements of a larger >>>>> list: >>>>> >>>>> >>>>>>>> comp[6] >>>>> ['6558', 'NM_001046.2', 'SLC12A2', '6037226', '2', 'chr5', >>>>> '127502453', >>>>> '127502454', 'het-ref', 'snp', 'A', 'T', 'A', '185', '113', '184', >>>>> '112', >>>>> 'VQHIGH', 'VQHIGH', '', '', '', '', '259974', '9', '6', '6', '15', >>>>> '6558:NM_001046.2:SLC12A2:CDS:MISSENSE', >>>>> '6558:NM_001046.2:SLC12A2:CDS:NO-CHANGE', 'PFAM:PF01490:Aa_trans', >>>>> '', >>>>> >>>>> >>>> '', >>>> >>>> >>>>> '', '0.99', '2', '0.99', '0.998', '1.01', '1.000', '0.5', '0.46', >>>>> '0.5', >>>>> '1', '18', '18', '19', 'ref-identical;onlyA', 'snp', '0.072', >>>>> '-1', >>>>> 'SQHIGH'] >>>>> >>>>> >>>>> >>>>> >>>>>>>> comp[7] >>>>> ['1302', 'NM_080679.2', 'COL11A2', '6525172', '2', 'chr6', >>>>> '33271374', >>>>> '33271376', 'het-ref', 'del', 'GT', '', 'GT', '542', '542', '458', >>>>> '458', >>>>> 'VQHIGH', 'VQHIGH', '', '', '', '', '71150', '34', '106', '106', >>>>> '140', >>>>> '1302:NM_080679.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC', >>>>> >>>>> >>>>> >>>> '1302:NM_080679.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC;1302:NM_080680.2 >>>> :COL >>>> 11A >>>> >>>> >>>>> 2:TSS-UPSTREAM:UNKNOWN-INC;1302:NM_080681.2:COL11A2:TSS-UPSTREAM: >>>>> UNKN >>>>> OWN- >>>>> INC;6257:NM_021976.3:RXRB:CDS:NO-CHANGE', >>>>> '', '', '', '', '0.95', '2', '0.98', '0.998', '0.99', '1.000', >>>>> '0.46', >>>>> '0.42', '0.5', '0', '102', '102', '102', 'ref-identical;onlyA', >>>>> 'del', >>>>> '0.990', '6', 'SQHIGH'] >>>>> >>>>> >>>>> >>>>> >>>>>>>> comp[8] >>>>> ['1302', 'NM_080680.2', 'COL11A2', '6525172', '2', 'chr6', >>>>> '33271374', >>>>> '33271376', 'het-ref', 'del', 'GT', '', 'GT', '542', '542', '458', >>>>> '458', >>>>> 'VQHIGH', 'VQHIGH', '', '', '', '', '71150', '34', '106', '106', >>>>> '140', >>>>> '1302:NM_080680.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC', >>>>> >>>>> >>>>> >>>> '1302:NM_080679.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC;1302:NM_080680.2 >>>> :COL >>>> 11A >>>> >>>> >>>>> 2:TSS-UPSTREAM:UNKNOWN-INC;1302:NM_080681.2:COL11A2:TSS-UPSTREAM: >>>>> UNKN >>>>> OWN- >>>>> INC;6257:NM_021976.3:RXRB:CDS:NO-CHANGE', >>>>> '', '', '', '', '0.95', '2', '0.98', '0.998', '0.99', '1.000', >>>>> '0.46', >>>>> '0.42', '0.5', '0', '102', '102', '102', 'ref-identical;onlyA', >>>>> 'del', >>>>> '0.990', '6', 'SQHIGH'] >>>>> >>>>> >>>>> >>>>>>>> >>>>> >>>>> ------ >>>>> Can we apply the dictionary method to the problem where the key of >>>>> the dictionary is the first element of the three smaller lists >>>> ('6558','1302', >>>> >>>> >>>>> '1302'). The second and third elements of the larger list >>>>> (starting >>>>> with '1302') need to be collapsed into a single element, based on >>>>> their second element ( 'NM_080679.2') and ('NM_080680.2') in a >>>>> way similar to how we had tackled the toy problem: >>>>> >>>>> x = [['cat', 'NM123', 12], ['cat', 'NM234', 12], ['dog', 'NM56', >>>>> 65]] >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> On Thu, Dec 5, 2013 at 4:18 AM, Michiel Overtoom >>>>> >>>>> wrote: >>>>> >>>>> >>>>> >>>>> >>>>>> >>>>>> On Dec 5, 2013, at 10:09, Vikram K wrote: >>>>>> >>>>>> >>>>>> >>>>>> >>>>>>> another option could have been to obtain a dictionary like >>>>>>> so: >>>>>>> {'dog': >>>>>>> ['NM56', 65], 'cat': ['NM123,NM234', 12]} >>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>>> Oh, in that case the code can become somewhat simpler: >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> x = [['cat', 'NM123', 12], ['cat', 'NM234', 12], ['dog', >>>>>> 'NM56', >>>>>> 65]] >>>>>> >>>>>> >>>>>> >>>>>> d = {} for key, label, quant in x: if key in d: d[key][0] += ", >>>>>> " + >>>>>> >>>>>> >>>> label >>>>>> else: >>>>>> d[key] = [label, quant] >>>>>> >>>>>> print d >>>>>> >>>>>> >>>>>> I agree with Michael that the problem is somewhat >>>>>> underspecified, but it's a starting point. >>>>>> >>>>>> Greetings, >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> "If you don't know, the thing to do is not to get scared, but to >>>>>> learn." - Ayn Rand >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>> _______________________________________________ >>>>> Baypiggies mailing list >>>>> Baypiggies at python.org >>>>> To change your subscription options or unsubscribe: >>>>> https://mail.python.org/mailman/listinfo/baypiggies >>>>> >>>>> >>>> >>>> >>>> >>> >> >> >> > From martin at falatic.com Thu Dec 5 13:52:35 2013 From: martin at falatic.com (Martin Falatic) Date: Thu, 5 Dec 2013 04:52:35 -0800 (PST) Subject: [Baypiggies] Fwd: manipulating lists question In-Reply-To: <64669.24.23.181.240.1386242465.squirrel@martin-wwwss5.ssl.supercp.com > References: <99D5C85E-3796-4056-95B4-928A9C8E85C9@xs4all.nl> <63483.24.23.181.240.1386238955.squirrel@martin-wwwss5.ssl.supercp.com> <63996.24.23.181.240.1386240714.squirrel@martin-wwwss5.ssl.supercp.com> <64669.24.23.181.240.1386242465.squirrel@martin-wwwss5.ssl.supercp.com> Message-ID: <1810.24.23.181.240.1386247955.squirrel@martin-wwwss5.ssl.supercp.com> Actually, I'll amend that: if you want to "compress" the data in this way you need to intelligently expand a given field into a list when conflicting data enters (this all assumes you cannot contain all the data in full-blown form prior to trying to compress it in this way - in effect, taking an input stream that might not be ordered by keys to begin with (the 1302 set comes in, then the 6558 set, then maybe way later another 1302 set...) If you could contain all of a given collection of data sets (say all the 1302s) in memory before processing them, it's relatively easy to expand the fields into lists when necessary. If you cannot, then there will need to be more code to expand a given flat field into a list. Suppose we have n=4 input lists: ['cat', 'NM123', 12] ['cat', 'NM123', 12] ['cat', 'NM123', 65] ['cat', 'NM456', 34] You want to end up with the following (step-wise): {'cat': ['NM123', 12]} {'cat': ['NM123', 12]} {'cat': ['NM123', [12,12,65]]} {'cat': [['NM123','NM123','NM123','NM456'], [12,12,65,34]]} It sounds easy at first but you have to keep track of how many unique data sets you have so far for a given key to properly inflate a given field. Again, this all assumes the object is a lossless compaction of a set of large and mostly-repetitive data lists. If mangling or discarding some of the data is OK then it's a simpler matter, but clearly there's a desire to retain the differing data and thus you probably need to be able to reconstruct it in the original order too. Note that if you have 30 sets of the same data in a given field for a given key and one if different you still get all 29 identical + 1 different values in a list for that field, but the other fields stay as single values, saving space. As long as the data sets for a given key are all unique you're being as efficient as possible without sacrificing data fidelity, and it's still human-readable. All that said, here's code that compacts your data without losing any of it (that is, you could reinflate each non-identical data set accurately). You can easily convert the sub-lists to comma/semicolon-delimited text strings as desired. I only left the debug prints in case you want to more quickly inspect the flow of how it works. *It worked with your inputs as well.* x = [['cat', 'NM123', 12], ['cat', 'NM123', 12], ['cat', 'NM123', 65], ['cat', 'NM456', 34], ['dog', 'NM56', 65]] y = dict() # Output values k = dict() # List of counts of a given key for parts in x: #print() key = parts[0] if key in y: k[key] += 1 else: k[key] = 1 y[key] = [] #print ("Processing key", key, k[key]) for idx in xrange(len(parts)-1): new_data = parts[idx+1] #print ("Processing index", idx, len(y[key]), new_data) if len(y[key]) <= idx: #print("new element") y[key].append(new_data) else: cur_data = y[key][idx] if type(cur_data) is list: #print("Appending to list") cur_data.append(new_data) else: if not cur_data == new_data: cur_data = [cur_data] * (k[key]-1) cur_data.append(new_data) y[key][idx] = cur_data #print("promoting elements to list", cur_data) else: pass # Not a contradicting data point print(x) print(y) for x = [['cat', 'NM123', 12], ['cat', 'NM123', 12], ['cat', 'NM123', 65], ['cat', 'NM456', 34], ['dog', 'NM56', 65]] I got y = {'dog': ['NM56', 65], 'cat': [['NM123', 'NM123', 'NM123', 'NM456'], [12, 12, 65, 34]]} Which matches what we would expect to see. - Marty On Thu, December 5, 2013 03:21, Martin Falatic wrote: > One can leave that as an exercise for the reader. :-) > > > I'm not sure why this gets a ';' versus a ',', nor is it clear if these > field lists are supposed to be deduped or ordered or what... consider if > you have three sets of data for 1302, and two ONLY vary by this field 28. > If you go to reconstruct the data set you end up with a somewhat mangled > thing. > > I take it this is an effort to compress the original data set to a more > manageable size for output / internal representation, which suggests > deduping isn't desirable (but which also suggests that you should simply > include every instance for fields 1 (which we already do) and 28, and > hope none of the other fields vary). > > On that note I'll throw this idea out there: given key field 0 as > identical for n sets of data, for every subsequent field [1:] if the item > is a str or int, consider it duplicated for all n sets. If the item is a > list then it much have exactly n elements (in the order the n sets were > parsed). That way if another fields is found to vary unexpectedly, it'll > simply become a list of n elements (many might be the same). > > You can always take that and stringify the elements and lists for storage > or whatever. The idea is that your internal data representation is a > much easier to work with set of lists/strs/ints within dictionary entries. > > > > - Marty > > > > On Thu, December 5, 2013 03:06, Vikram K wrote: > >> Good catch. All the other elements remain the same except this one. >> Element >> 28 needs to be changed (in the merged/collapsed list) so that when we >> fuse or merge two elements of the larger list into one then Element 28 >> of the new element is (just combine whatever is present in element 28 in >> both the lists keeping a ';' as delimiter): >> >> '1302:NM_080680.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC'; ' >> 1302:NM_080679.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC' >> >> >> >> >> >> On Thu, Dec 5, 2013 at 5:51 AM, Martin Falatic >> wrote: >> >> >> >>> My solution works for the first three elements as stated, but what >>> you do with the rest of the elements is tricky if they differ for a >>> given key. >>> >>> For 1302 all the fields in the slice [3:] match each other *except* >>> element 28: '1302:NM_080679.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC' >>> '1302:NM_080680.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC' >>> >>> >>> >>> Does this potentially happen with other elements at times? At this >>> point you're faced with either discarding data or mangling data >>> together. The "collapse" just takes the last [3:] slice encountered >>> (for >>> that remainder of data). Is that acceptable? >>> >>> - Marty >>> >>> >>> >>> >>> On Thu, December 5, 2013 02:33, Vikram K wrote: >>> >>> >>>> In the example i have given, the second and third elements of the >>>> larger list (comp[7] and comp[8]) have a 1:1 mapping after the >>>> second element. >>> So >>> >>> >>>> i would like to keep the first element as it is and then collapse >>>> or merge the second and third elements (comp[7] and comp[8]) into a >>>> single element: >>>> >>>> >>>> >>>>>>> comp[6] >>>> ['6558', 'NM_001046.2', 'SLC12A2', '6037226', '2', 'chr5', >>>> '127502453', >>>> '127502454', 'het-ref', 'snp', 'A', 'T', 'A', '185', '113', '184', >>>> '112', >>>> 'VQHIGH', 'VQHIGH', '', '', '', '', '259974', '9', '6', '6', '15', >>>> '6558:NM_001046.2:SLC12A2:CDS:MISSENSE', >>>> '6558:NM_001046.2:SLC12A2:CDS:NO-CHANGE', 'PFAM:PF01490:Aa_trans', >>>> '', >>>> >>>> >>> '', >>> >>> >>>> '', '0.99', '2', '0.99', '0.998', '1.01', '1.000', '0.5', '0.46', >>>> '0.5', >>>> '1', '18', '18', '19', 'ref-identical;onlyA', 'snp', '0.072', '-1', >>>> 'SQHIGH'] >>>> >>>> >>>> >>>> >>>>>>> comp[7] >>>> ['1302', 'NM_080679.2', 'COL11A2', '6525172', '2', 'chr6', >>>> '33271374', >>>> '33271376', 'het-ref', 'del', 'GT', '', 'GT', '542', '542', '458', >>>> '458', >>>> 'VQHIGH', 'VQHIGH', '', '', '', '', '71150', '34', '106', '106', >>>> '140', >>>> '1302:NM_080679.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC', >>>> >>>> >>>> >>> '1302:NM_080679.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC;1302:NM_080680.2:C >>> OL >>> 11A >>> >>> >>>> 2:TSS-UPSTREAM:UNKNOWN-INC;1302:NM_080681.2:COL11A2:TSS-UPSTREAM:UN >>>> KN >>>> OWN- >>>> INC;6257:NM_021976.3:RXRB:CDS:NO-CHANGE', >>>> '', '', '', '', '0.95', '2', '0.98', '0.998', '0.99', '1.000', >>>> '0.46', >>>> '0.42', '0.5', '0', '102', '102', '102', 'ref-identical;onlyA', >>>> 'del', >>>> '0.990', '6', 'SQHIGH'] >>>> >>>> >>>> >>>> >>>>>>> comp[8] >>>> ['1302', 'NM_080680.2', 'COL11A2', '6525172', '2', 'chr6', >>>> '33271374', >>>> '33271376', 'het-ref', 'del', 'GT', '', 'GT', '542', '542', '458', >>>> '458', >>>> 'VQHIGH', 'VQHIGH', '', '', '', '', '71150', '34', '106', '106', >>>> '140', >>>> '1302:NM_080680.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC', >>>> >>>> >>>> >>> '1302:NM_080679.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC;1302:NM_080680.2:C >>> OL >>> 11A >>> >>> >>>> 2:TSS-UPSTREAM:UNKNOWN-INC;1302:NM_080681.2:COL11A2:TSS-UPSTREAM:UN >>>> KN >>>> OWN- >>>> INC;6257:NM_021976.3:RXRB:CDS:NO-CHANGE', >>>> '', '', '', '', '0.95', '2', '0.98', '0.998', '0.99', '1.000', >>>> '0.46', >>>> '0.42', '0.5', '0', '102', '102', '102', 'ref-identical;onlyA', >>>> 'del', >>>> '0.990', '6', 'SQHIGH'] >>>> >>>> >>>> >>>> >>>> After collapsing comp[7] and comp[8] i get: >>>> >>>> >>>> >>>> >>>>>>> collapsed = ['1302', 'NM_080679.2,NM_080680.2', 'COL11A2', >>>>>>> '6525172', >>>>>>> >>>>>>> >>>>>>> >>>> '2', 'chr6', '33271374', '33271376', 'het-ref', 'del', 'GT', '', >>>> 'GT', >>>> '542', '542', '458', '458', 'VQHIGH', 'VQHIGH', '', '', '', '', >>>> '71150', >>>> '34', '106', '106', '140', >>>> '1302:NM_080680.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC', >>>> >>>> >>>> >>> '1302:NM_080679.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC;1302:NM_080680.2:C >>> OL >>> 11A >>> >>> >>>> 2:TSS-UPSTREAM:UNKNOWN-INC;1302:NM_080681.2:COL11A2:TSS-UPSTREAM:UN >>>> KN >>>> OWN- >>>> INC;6257:NM_021976.3:RXRB:CDS:NO-CHANGE', >>>> '', '', '', '', '0.95', '2', '0.98', '0.998', '0.99', '1.000', >>>> '0.46', >>>> '0.42', '0.5', '0', '102', '102', '102', 'ref-identical;onlyA', >>>> 'del', >>>> '0.990', '6', 'SQHIGH'] >>>> >>>> >>>> >>>> >>>> So in my larger list, after the modification, comp[6] is the first >>>> element and collapsed the second element. >>>>>>> >>>> >>>> >>>> On Thu, Dec 5, 2013 at 5:22 AM, Martin Falatic >>>> wrote: >>>> >>>> >>>> >>>> >>>>> Ah, genetics! Intriguing... >>>>> >>>>> >>>>> >>>>> >>>>> Do you need anything beyond the third elements of each list? Does >>>>> the third element always map 1:1 with the first, or could it >>>>> vary? If so, >>>>> what then? >>>>> >>>>> To refer to the simplified example, could you have this? >>>>> x = [['cat', 'NM123', 12], ['cat', 'NM234', 43], ['dog', 'NM56', >>>>> 65]] >>>>> >>>>> >>>>> >>>>> If so, what is the expected output? >>>>> >>>>> >>>>> >>>>> >>>>> - Marty >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> On Thu, December 5, 2013 02:11, Vikram K wrote: >>>>> >>>>> >>>>> >>>>>> i am having some difficulty in applying this to my actual >>>>>> problem although i love the dictionary method. Imagine the >>>>>> following three lists are the first, second and third elements >>>>>> of a larger list: >>>>>> >>>>>> >>>>>> >>>>>>>>> comp[6] >>>>>> ['6558', 'NM_001046.2', 'SLC12A2', '6037226', '2', 'chr5', >>>>>> '127502453', >>>>>> '127502454', 'het-ref', 'snp', 'A', 'T', 'A', '185', '113', >>>>>> '184', >>>>>> '112', >>>>>> 'VQHIGH', 'VQHIGH', '', '', '', '', '259974', '9', '6', '6', >>>>>> '15', >>>>>> '6558:NM_001046.2:SLC12A2:CDS:MISSENSE', >>>>>> '6558:NM_001046.2:SLC12A2:CDS:NO-CHANGE', >>>>>> 'PFAM:PF01490:Aa_trans', >>>>>> '', >>>>>> >>>>>> >>>>>> >>>>> '', >>>>> >>>>> >>>>> >>>>>> '', '0.99', '2', '0.99', '0.998', '1.01', '1.000', '0.5', >>>>>> '0.46', >>>>>> '0.5', >>>>>> '1', '18', '18', '19', 'ref-identical;onlyA', 'snp', '0.072', >>>>>> '-1', >>>>>> 'SQHIGH'] >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>>>>> comp[7] >>>>>> ['1302', 'NM_080679.2', 'COL11A2', '6525172', '2', 'chr6', >>>>>> '33271374', >>>>>> '33271376', 'het-ref', 'del', 'GT', '', 'GT', '542', '542', >>>>>> '458', >>>>>> '458', >>>>>> 'VQHIGH', 'VQHIGH', '', '', '', '', '71150', '34', '106', '106', >>>>>> '140', >>>>>> '1302:NM_080679.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC', >>>>>> >>>>>> >>>>>> >>>>>> >>>>> '1302:NM_080679.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC;1302:NM_080680 >>>>> .2 >>>>> :COL >>>>> 11A >>>>> >>>>> >>>>> >>>>>> 2:TSS-UPSTREAM:UNKNOWN-INC;1302:NM_080681.2:COL11A2:TSS-UPSTREA >>>>>> M: >>>>>> UNKN >>>>>> OWN- >>>>>> INC;6257:NM_021976.3:RXRB:CDS:NO-CHANGE', >>>>>> '', '', '', '', '0.95', '2', '0.98', '0.998', '0.99', '1.000', >>>>>> '0.46', >>>>>> '0.42', '0.5', '0', '102', '102', '102', 'ref-identical;onlyA', >>>>>> 'del', >>>>>> '0.990', '6', 'SQHIGH'] >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>>>>> comp[8] >>>>>> ['1302', 'NM_080680.2', 'COL11A2', '6525172', '2', 'chr6', >>>>>> '33271374', >>>>>> '33271376', 'het-ref', 'del', 'GT', '', 'GT', '542', '542', >>>>>> '458', >>>>>> '458', >>>>>> 'VQHIGH', 'VQHIGH', '', '', '', '', '71150', '34', '106', '106', >>>>>> '140', >>>>>> '1302:NM_080680.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC', >>>>>> >>>>>> >>>>>> >>>>>> >>>>> '1302:NM_080679.2:COL11A2:TSS-UPSTREAM:UNKNOWN-INC;1302:NM_080680 >>>>> .2 >>>>> :COL >>>>> 11A >>>>> >>>>> >>>>> >>>>>> 2:TSS-UPSTREAM:UNKNOWN-INC;1302:NM_080681.2:COL11A2:TSS-UPSTREA >>>>>> M: >>>>>> UNKN >>>>>> OWN- >>>>>> INC;6257:NM_021976.3:RXRB:CDS:NO-CHANGE', >>>>>> '', '', '', '', '0.95', '2', '0.98', '0.998', '0.99', '1.000', >>>>>> '0.46', >>>>>> '0.42', '0.5', '0', '102', '102', '102', 'ref-identical;onlyA', >>>>>> 'del', >>>>>> '0.990', '6', 'SQHIGH'] >>>>>> >>>>>> >>>>>> >>>>>> >>>>>>>>> >>>>>> >>>>>> ------ >>>>>> Can we apply the dictionary method to the problem where the key >>>>>> of the dictionary is the first element of the three smaller >>>>>> lists >>>>> ('6558','1302', >>>>> >>>>> >>>>> >>>>>> '1302'). The second and third elements of the larger list >>>>>> (starting >>>>>> with '1302') need to be collapsed into a single element, based >>>>>> on their second element ( 'NM_080679.2') and ('NM_080680.2') in >>>>>> a way similar to how we had tackled the toy problem: >>>>>> >>>>>> x = [['cat', 'NM123', 12], ['cat', 'NM234', 12], ['dog', >>>>>> 'NM56', >>>>>> 65]] >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> On Thu, Dec 5, 2013 at 4:18 AM, Michiel Overtoom >>>>>> >>>>>> wrote: >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>>> >>>>>>> On Dec 5, 2013, at 10:09, Vikram K wrote: >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>>> another option could have been to obtain a dictionary like >>>>>>>> so: >>>>>>>> {'dog': >>>>>>>> ['NM56', 65], 'cat': ['NM123,NM234', 12]} >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> Oh, in that case the code can become somewhat simpler: >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> x = [['cat', 'NM123', 12], ['cat', 'NM234', 12], ['dog', >>>>>>> 'NM56', >>>>>>> 65]] >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> d = {} for key, label, quant in x: if key in d: d[key][0] += >>>>>>> ", >>>>>>> " + >>>>>>> >>>>>>> >>>>>>> >>>>> label >>>>>>> else: >>>>>>> d[key] = [label, quant] >>>>>>> >>>>>>> print d >>>>>>> >>>>>>> >>>>>>> I agree with Michael that the problem is somewhat >>>>>>> underspecified, but it's a starting point. >>>>>>> >>>>>>> Greetings, >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> "If you don't know, the thing to do is not to get scared, but >>>>>>> to learn." - Ayn Rand >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>> _______________________________________________ >>>>>> Baypiggies mailing list >>>>>> Baypiggies at python.org >>>>>> To change your subscription options or unsubscribe: >>>>>> https://mail.python.org/mailman/listinfo/baypiggies >>>>>> >>>>>> >>>>>> >>>>> >>>>> >>>>> >>>> >>> >>> >>> >> > > > _______________________________________________ > Baypiggies mailing list > Baypiggies at python.org > To change your subscription options or unsubscribe: > https://mail.python.org/mailman/listinfo/baypiggies > > From fil at macfellow.com Thu Dec 5 10:12:53 2013 From: fil at macfellow.com (Filip Machi) Date: Thu, 5 Dec 2013 01:12:53 -0800 Subject: [Baypiggies] manipulating lists question In-Reply-To: References: Message-ID: [I'm not sure I can post to BayPiggies using this email address.] This is a start, y1 and y2 the two forms noted by Michael: #!/usr/bin/env python2.7 from collections import OrderedDict x = [['cat','NM123',12],['cat','NM234',12], ['dog', 'NM56',65]] z = OrderedDict([(e[0], (set(), set())) for e in x]) for e in x: z[e[0]][0].update([e[1]]) z[e[0]][1].update([e[2]]) y1 = [[e] + list(z[e][0]) + list(z[e][1]) for e in z] print y1 y2 = [[e] + [', '.join(list(z[e][0]))] + list(z[e][1]) for e in z] print y2 Output: [['cat', 'NM234', 'NM123', 12], ['dog', 'NM56', 65]] [['cat', 'NM234, NM123', 12], ['dog', 'NM56', 65]] -Fil On Dec 5, 2013, at 00:53 , Michiel Overtoom wrote: > > On Dec 5, 2013, at 09:06, Vikram K wrote: > >> Any suggestions on what i have to do to go from x to y? >> >>>>> x = [['cat','NM123',12],['cat','NM234',12], ['dog', 'NM56',65]] >>>>> y = [['cat','NM123, NM234', 12], ['dog', 'NM56', 65]] > > Are you sure you mean > > [['cat','NM123, NM234', 12], ['dog', 'NM56', 65]] > > and not > > [['cat','NM123', 'NM234', 12], ['dog', 'NM56', 65]] > > ? > > >>>>> >> >> Regards >> Vik >> _______________________________________________ >> Baypiggies mailing list >> Baypiggies at python.org >> To change your subscription options or unsubscribe: >> https://mail.python.org/mailman/listinfo/baypiggies > > > -- > Veel spullen zijn tegenwoordig waardeloos, maar toch hecht men er > veel waarde aan. Mensen hebben vaak teveel spullen. Hoe komt dat en > wat doe je ertegen? - een essay van Paul Graham, op http://www.michielovertoom.com/articles/paul-graham-spullen > > "Een prima telefoon met draaischijf weggooien? Die zou ik nog wel > eens nodig kunnen hebben!" > > > _______________________________________________ > Baypiggies mailing list > Baypiggies at python.org > To change your subscription options or unsubscribe: > https://mail.python.org/mailman/listinfo/baypiggies From aahz at pythoncraft.com Thu Dec 5 16:06:55 2013 From: aahz at pythoncraft.com (Aahz) Date: Thu, 5 Dec 2013 07:06:55 -0800 Subject: [Baypiggies] ADMIN: multiple e-mail addresses Message-ID: <20131205150655.GA5563@panix.com> On Thu, Dec 05, 2013, Filip Machi wrote: > > [I'm not sure I can post to BayPiggies using this email address.] You can always check your subscriptions at https://mail.python.org/mailman/listinfo/baypiggies If you have multiple addresses, you should subscribe ALL of the ones you're likely to post from. You can then set all but one to "nomail". Otherwise, the list admins have to rescue your post from moderation. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ A house is a machine to keep your cat dry. From mvoorhie at yahoo.com Thu Dec 5 18:16:50 2013 From: mvoorhie at yahoo.com (Mark Voorhies) Date: Thu, 05 Dec 2013 09:16:50 -0800 Subject: [Baypiggies] Fwd: manipulating lists question In-Reply-To: References: <99D5C85E-3796-4056-95B4-928A9C8E85C9@xs4all.nl> Message-ID: <52A0B502.60303@yahoo.com> On 12/05/2013 02:11 AM, Vikram K wrote: > i am having some difficulty in applying this to my actual problem although > i love the dictionary method. Imagine the following three lists are the > first, second and third elements of a larger list: > >>>> comp[6] > ['6558', 'NM_001046.2', 'SLC12A2', '6037226', '2', 'chr5', '127502453', > '127502454', 'het-ref', 'snp', 'A', 'T', 'A', '185', '113', '184', '112', > 'VQHIGH', 'VQHIGH', '', '', '', '', '259974', '9', '6', '6', '15', > '6558:NM_001046.2:SLC12A2:CDS:MISSENSE', > '6558:NM_001046.2:SLC12A2:CDS:NO-CHANGE', 'PFAM:PF01490:Aa_trans', '', '', > '', '0.99', '2', '0.99', '0.998', '1.01', '1.000', '0.5', '0.46', '0.5', > '1', '18', '18', '19', 'ref-identical;onlyA', 'snp', '0.072', '-1', > 'SQHIGH'] This looks like the result of a SQL query on a database of SNPs. If so, you can ask the database to solve your de-duplication problem for you. E.g., with a MySQL query like: SELECT id, GROUP_CONCAT(DISTINCT accession), GROUP_CONCAT(DISTINCT name), etc FROM ... WHERE ... GROUP BY id; You can almost always get a more useful result by including more context in your query. --Mark From nicholsonjf at gmail.com Thu Dec 5 20:22:51 2013 From: nicholsonjf at gmail.com (James Nicholson) Date: Thu, 5 Dec 2013 11:22:51 -0800 Subject: [Baypiggies] Fwd: Big-O talk is up on YouTube In-Reply-To: References: Message-ID: Hi All, I tried sending the below message to the group on 11/30 but there was an issue with my list membership. Let me know if you have any issues viewing the video. -James ---------- Forwarded message ---------- From: *James Nicholson* Date: Saturday, November 30, 2013 Subject: Big-O talk is up on YouTube To: Baypiggies Cc: Glen Jarvis Hey Everyone, The video of Jeremy Holman's talk about the Big-O can be viewed here: http://youtu.be/1xCSqMbxd_k Also, please subscribe to our Bay Piggies channel on YouTube! If we get 100 subscribers, Google will allow us to live stream ;) Happy holidays! Cheers, James -------------- next part -------------- An HTML attachment was scrubbed... URL: From abhishek.vit at gmail.com Thu Dec 5 20:58:47 2013 From: abhishek.vit at gmail.com (Abhishek Pratap) Date: Thu, 5 Dec 2013 11:58:47 -0800 Subject: [Baypiggies] Fwd: Big-O talk is up on YouTube In-Reply-To: References: Message-ID: This is great..after moving to Seattle I miss these meetings big time..seeing it virtually will be able to bring some sort of closure I guess.. +++1 for subscribe and I say we all do it. Cheers! -Abhi On Thu, Dec 5, 2013 at 11:22 AM, James Nicholson wrote: > Hi All, > > I tried sending the below message to the group on 11/30 but there was an > issue with my list membership. > > Let me know if you have any issues viewing the video. > > -James > > ---------- Forwarded message ---------- > From: James Nicholson > Date: Saturday, November 30, 2013 > Subject: Big-O talk is up on YouTube > To: Baypiggies > Cc: Glen Jarvis > > > Hey Everyone, > > The video of Jeremy Holman's talk about the Big-O can be viewed here: > > http://youtu.be/1xCSqMbxd_k > > Also, please subscribe to our Bay Piggies channel on YouTube! If we get 100 > subscribers, Google will allow us to live stream ;) > > Happy holidays! > > Cheers, > James > > > > _______________________________________________ > Baypiggies mailing list > Baypiggies at python.org > To change your subscription options or unsubscribe: > https://mail.python.org/mailman/listinfo/baypiggies From annaraven at gmail.com Mon Dec 16 22:52:23 2013 From: annaraven at gmail.com (Anna Ravenscroft) Date: Mon, 16 Dec 2013 13:52:23 -0800 Subject: [Baypiggies] Fwd: Re: [oscon] OSCON 2014 CfP is out! In-Reply-To: <2102979186.9687815.1387222359140.JavaMail.root@oreilly.com> References: <2102979186.9687815.1387222359140.JavaMail.root@oreilly.com> Message-ID: >From my phone. ---------- Forwarded message ---------- From: "Simon St.Laurent" Date: Dec 16, 2013 11:32 AM Subject: Re: [oscon] OSCON 2014 CfP is out! To: "Michael Bernstein" Cc: "OSCON Committee" Thanks, Michael! I should have announced this here Friday when it went up. Somehow I blanked on that. Please definitely let folks know we're looking for (great) talks, maybe planting seeds before the holidays? I'm very happy that the call is open through the end of January, so there should be time after the holiday to encourage people to put in concrete proposals. Thanks, everyone! More soon! Thanks, Simon ----- Original Message ----- > > > > I see the CfP is out: http://www.oscon.com/oscon2014/public/cfp/308 > > I've had a bit of personal turmoil the past few years, particularly > at this time of year, that caused me to neglect my commitment to > getting the word out on the CfP to the communities I am involved in. > This year (despite some similar turmoil) will be different. > > > Starting today I'll be doing outreach to get high-quality proposals > submitted early, and *will* make sure I do my share of reviewing as > well. > > > Cheers, > > > -- > Michael R. Bernstein > michaelbernstein.com > _______________________________________________ > Oscon mailing list > Oscon at oscon.oreilly.com > http://oscon.oreilly.com/mailman/listinfo/oscon_oscon.oreilly.com > _______________________________________________ Oscon mailing list Oscon at oscon.oreilly.com http://oscon.oreilly.com/mailman/listinfo/oscon_oscon.oreilly.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From aahz at pythoncraft.com Thu Dec 19 18:23:37 2013 From: aahz at pythoncraft.com (Aahz) Date: Thu, 19 Dec 2013 09:23:37 -0800 Subject: [Baypiggies] FWD: CS Faculty Position open at Foothill College Message-ID: <20131219172337.GA17262@panix.com> Forwarding with permission; Elaine is a former listmember but is not currently subscribed (and yeah, I'm breaking the job posting rules for once, sue me): ----- Forwarded message from Elaine Haight ----- > Date: Wed, 18 Dec 2013 23:01:17 +0000 > From: Elaine Haight > To: Elaine Haight > Subject: CS Faculty Position open at Foothill College > > Dear Friends of Computer Science at Foothill, > > I am writing to let you know that, for the first time in ten years, we > have a full time tenure track faculty position open in the computer > science department here at Foothill. I was hoping that you or someone > you know may be interested in applying for the job. Applications will > be accepted until 27 Jan 2014. A Master's degree in a technical field > is required. > > It is critically important to the economic vitality of the Silicon > Valley that we educate the next generation of engineers and > technicians. You can help us a lot by forwarding the following link to > anyone in your network who may be interested: > > https://ch.tbe.taleo.net/CH17/ats/careers/requisition.jsp?org=FHDA&cws=1&rid=838 > > Thanks so much and have a great holiday! > > -Elaine Haight, CS Faculty, Foothill College, Los Altos Hills, CA ----- End forwarded message ----- From jwegis at gmail.com Thu Dec 19 19:16:02 2013 From: jwegis at gmail.com (John Wegis) Date: Thu, 19 Dec 2013 10:16:02 -0800 Subject: [Baypiggies] Baypiggies Meeting Tonight @LInkedIn - Learning Math and Science Using Python Message-ID: Hello Everyone, Our presentation tonight will be "Learning Math and Science Using Python" by Peter Farrell. *Location:* *LinkedIn Corporation2061 Stierlin Ct (aka Bldg 6)Room - Neon CarrotMountain View, CA 94043* *Abstract:* In 1980 Seymour Papert?s *Mindstorms* suggested that everything meaningful in math education could be done using a computer. Mathematics teaching has yet to take full advantage of modern technology. Python should be in the toolbox of every student right from the beginning of algebra, not only because it makes arithmetic trivial but because it can facilitate deeper exploration of topics ranging from algebra to calculus. In this presentation we?ll see how Python can be used to clarify difficult topics high school students are faced with in math and science. Unlike outdated skills like factoring polynomials, working with functions is more valuable than ever thanks to computers and Python in particular because (forgive my oversimplification) in Python ?everything is a function.? High school math and physics courses also demand a knowledge of vectors, so more teachers should take advantage of the 3-D possibilities of Visual Python to model situations using vectors. It will be shown how physical situations otherwise involving disheartening differential equations can be modeled and easily solved in VPython. Furthermore, conjectures which were previously difficult to prove in physics can be easily created and visually proven by high school students with a little Python. *Speaker: * Peter Farrell has been a high school math teacher and tutor on the East Coast and the West Coast and even on the Equator. He?s currently a STEM Instructor at four Summit Charter High Schools in the Bay Area through Learningtech.org *If you wish to post jobs here, please go to : Job Listings * *Meeting Schedule* The meeting begins @ 7:30PM. The main presentation will start @ 7:40 PM, after the usual announcements. -------------- next part -------------- An HTML attachment was scrubbed... URL: From nicholsonjf at gmail.com Tue Dec 31 16:34:07 2013 From: nicholsonjf at gmail.com (James Nicholson) Date: Tue, 31 Dec 2013 07:34:07 -0800 Subject: [Baypiggies] Baypiggies Meeting Tonight @LInkedIn - Learning Math and Science Using Python In-Reply-To: References: Message-ID: The video from Peter's talk is now up on our YouTube channel, and can be found here: http://youtu.be/HfvQ6O7Di0g -James On Thursday, December 19, 2013, John Wegis wrote: > Hello Everyone, > > Our presentation tonight will be "Learning Math and Science Using Python" > by Peter Farrell. > > *Location:* > > > > > *LinkedIn Corporation2061 Stierlin Ct (aka Bldg 6)Room - Neon > CarrotMountain View, CA 94043* > > *Abstract:* > > In 1980 Seymour Papert?s *Mindstorms* suggested that everything > meaningful in math education could be done using a computer. Mathematics > teaching has yet to take full advantage of modern technology. > > Python should be in the toolbox of every student right from the beginning > of algebra, not only because it makes arithmetic trivial but because it can > facilitate deeper exploration of topics ranging from algebra to calculus. > > In this presentation we?ll see how Python can be used to clarify difficult > topics high school students are faced with in math and science. > > Unlike outdated skills like factoring polynomials, working with functions > is more valuable than ever thanks to computers and Python in particular > because (forgive my oversimplification) in Python ?everything is a > function.? > > High school math and physics courses also demand a knowledge of vectors, > so more teachers should take advantage of the 3-D possibilities of Visual > Python to model situations using vectors. It will be shown how physical > situations otherwise involving disheartening differential equations can be > modeled and easily solved in VPython. Furthermore, conjectures which were > previously difficult to prove in physics can be easily created and visually > proven by high school students with a little Python. > *Speaker: * > Peter Farrell has been a high school math teacher and tutor on the East > Coast and the West Coast and even on the Equator. He?s currently a STEM > Instructor at four Summit Charter High Schools in the Bay Area through > Learningtech.org > > > > *If you wish to post jobs here, please go to : Job Listings > * > > *Meeting Schedule* > The meeting begins @ 7:30PM. > The main presentation will start @ 7:40 PM, after the usual announcements. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From glen at glenjarvis.com Tue Dec 31 19:16:01 2013 From: glen at glenjarvis.com (Glen Jarvis) Date: Tue, 31 Dec 2013 10:16:01 -0800 Subject: [Baypiggies] Baypiggies Meeting Tonight @LInkedIn - Learning Math and Science Using Python In-Reply-To: References: Message-ID: I just want to throw this out there: James has been doing an *incredible* job being our videographer. In fact, he has some very exciting ideas that would improve the process greatly (and they seem very realistic/plausible). KUDOS, James! Cheers, Glen On Tue, Dec 31, 2013 at 7:34 AM, James Nicholson wrote: > The video from Peter's talk is now up on our YouTube channel, and can be > found here: > > http://youtu.be/HfvQ6O7Di0g > > -James > > > On Thursday, December 19, 2013, John Wegis wrote: > >> Hello Everyone, >> >> Our presentation tonight will be "Learning Math and Science Using Python" >> by Peter Farrell. >> >> *Location:* >> >> >> >> >> *LinkedIn Corporation2061 Stierlin Ct (aka Bldg 6)Room - Neon >> CarrotMountain View, CA 94043* >> >> *Abstract:* >> >> In 1980 Seymour Papert?s *Mindstorms* suggested that everything >> meaningful in math education could be done using a computer. Mathematics >> teaching has yet to take full advantage of modern technology. >> >> Python should be in the toolbox of every student right from the beginning >> of algebra, not only because it makes arithmetic trivial but because it can >> facilitate deeper exploration of topics ranging from algebra to calculus. >> >> In this presentation we?ll see how Python can be used to clarify >> difficult topics high school students are faced with in math and science. >> >> Unlike outdated skills like factoring polynomials, working with functions >> is more valuable than ever thanks to computers and Python in particular >> because (forgive my oversimplification) in Python ?everything is a >> function.? >> >> High school math and physics courses also demand a knowledge of vectors, >> so more teachers should take advantage of the 3-D possibilities of Visual >> Python to model situations using vectors. It will be shown how physical >> situations otherwise involving disheartening differential equations can be >> modeled and easily solved in VPython. Furthermore, conjectures which >> were previously difficult to prove in physics can be easily created and >> visually proven by high school students with a little Python. >> *Speaker: * >> Peter Farrell has been a high school math teacher and tutor on the East >> Coast and the West Coast and even on the Equator. He?s currently a STEM >> Instructor at four Summit Charter High Schools in the Bay Area through >> Learningtech.org >> >> >> >> *If you wish to post jobs here, please go to : Job Listings >> * >> >> *Meeting Schedule* >> The meeting begins @ 7:30PM. >> The main presentation will start @ 7:40 PM, after the usual announcements. >> > > _______________________________________________ > Baypiggies mailing list > Baypiggies at python.org > To change your subscription options or unsubscribe: > https://mail.python.org/mailman/listinfo/baypiggies > -- We should be taught not to wait for inspiration to start a thing. Action always generates inspiration. Inspiration seldom generates action. -- Frank Tibolt -------------- next part -------------- An HTML attachment was scrubbed... URL: