[Tutor] Tutor Digest, Vol 142, Issue 11
Crusier
crusier at gmail.com
Mon Dec 14 11:16:39 EST 2015
Dear Alan,
Thank you very much for answering the question. If you don't mind,
please kindly let me know which library I should focus on among
beautifulsoup, selenium + PhantomJS, and dryscrape.
Have a good day
regards,
Hank
On Sun, Dec 13, 2015 at 5:11 PM, <tutor-request at python.org> wrote:
> Send Tutor mailing list submissions to
> tutor at python.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
> https://mail.python.org/mailman/listinfo/tutor
> or, via email, send a message with subject or body 'help' to
> tutor-request at python.org
>
> You can reach the person managing the list at
> tutor-owner at python.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of Tutor digest..."
>
>
> Today's Topics:
>
> 1. Re: Calculation error with a simple program (Steven D'Aprano)
> 2. Re: Calculation error with a simple program (Jim Gallaher)
> 3. Re: Calculation error with a simple program (Todd Purple)
> 4. Re: Tutor Digest, Vol 142, Issue 10 (Ken Hammer)
> 5. Beautiful Soup (Crusier)
> 6. Re: Beautiful Soup (Alan Gauld)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Sun, 13 Dec 2015 04:00:25 +1100
> From: Steven D'Aprano <steve at pearwood.info>
> To: tutor at python.org
> Subject: Re: [Tutor] Calculation error with a simple program
> Message-ID: <20151212170025.GK3821 at ando.pearwood.info>
> Content-Type: text/plain; charset=us-ascii
>
> On Sat, Dec 12, 2015 at 01:03:05AM -0600, Jim Gallaher wrote:
>> Hi everyone. I'm reading through a beginners Python book and came up
>> with a super simple program. I'm not getting any errors and everything
>> runs through, but there's a logical calculation error. What the program
>> does is take an amount and calculate a couple percentages and add a
>> couple fees.
>>
>> For example, if I put in a value of 1, it will output 752.12 as the sub
>> total and 753.12 as the grand total. It's off by 1 on sub total and 2 on
>> grand total.
>
> Check your arithmetic -- with a base price of $1, the sub total is
> $752.12 and the grand total is $753.12, exactly as Python calculates.
>
>
> But I wonder whether you have made a logic mistake in your code. You
> say:
>
> dealerPrep = basePrice + 500
> destinationCharge = basePrice + 250
>
> This means that the car will cost more than TRIPLE the base price.
> Instead of $1, enter a base price of $40,000 and you will see what I
> mean: now the dealer prep is $40500 and the destination charge is
> $40250, which added to the base price gives $120750 (plus tax and
> licence). Surely that's not right, the deal charges more than the cost
> of the car for preparation? I think what you want is:
>
> dealerPrep = 500
> destinationCharge = 250
>
>
>
> --
> Steve
>
>
> ------------------------------
>
> Message: 2
> Date: Sat, 12 Dec 2015 11:34:27 -0600
> From: Jim Gallaher <jcgallaher78 at gmail.com>
> To: tutor at python.org
> Subject: Re: [Tutor] Calculation error with a simple program
> Message-ID: <8774A398-4A0F-41DF-BBD2-941B49A3CC90 at gmail.com>
> Content-Type: text/plain; charset=us-ascii
>
> Hi Alan,
>
> I'm 100 percent sure I'm wrong. :-) I verified it when I fixed the mistake. The problem was it was adding in the basePrice and the fixed rates/percentages each time. So I figured it out when Ian said something about that.
>
> Thanks for everyone's help! :-)
>
> ------------------------------
>
> Message: 3
> Date: Sat, 12 Dec 2015 08:13:23 -0500
> From: Todd Purple <todd_purple at yahoo.com>
> To: Jim Gallaher <jcgallaher78 at gmail.com>
> Cc: tutor at python.org
> Subject: Re: [Tutor] Calculation error with a simple program
> Message-ID: <6EC6D759-3C19-4A6D-8DD7-6EFAE1958CCD at yahoo.com>
> Content-Type: text/plain; charset=us-ascii
>
>
>
>> On Dec 12, 2015, at 2:03 AM, Jim Gallaher <jcgallaher78 at gmail.com> wrote:
>>
>> Hi everyone. I'm reading through a beginners Python book and came up with a super simple program. I'm not getting any errors and everything runs through, but there's a logical calculation error. What the program does is take an amount and calculate a couple percentages and add a couple fees.
>>
>> For example, if I put in a value of 1, it will output 752.12 as the sub total and 753.12 as the grand total. It's off by 1 on sub total and 2 on grand total.
>>
>> Thanks in advance! Jim Gallaher
>>
>> # Car Salesman Calculator
>>
>> # User enters the base price of the car and the program adds tax, license, dealer prep, and destination charge.
>>
>> print("Car Sales Calculator")
>> basePrice = int(input("Please enter in the price of the car: "))
>>
>> # Misc charges to be added to the total cost of the car
>> tax = basePrice * .07
>> license = basePrice * .05
>> dealerPrep = basePrice + 500
>> destinationCharge = basePrice + 250
>>
>
> I think your main problem is right here. Why do dealerPrep and destinationCharge include the basePrice in their equation? This will add the basePrice in multiple times. I would simply set it like this:
>
> dealerPrep = 500
> destinationCharge = 250
>
> If these charges ever change, you can put in some sort of equation. However, it looks like these are meant to be a flat fee.
>
>> # Add the total misc charges together
>> subTotal = float(tax + license + dealerPrep + destinationCharge)
>>
>> # Add all the misic charges and include the base pricem
>> grandTotal = float(subTotal + basePrice)
>>
>> # Display the results
>> print("\nThe sub total is", subTotal)
>> print("\nYour grand Total is", grandTotal)
>>
>> input("\nPress the enter key to close the program.")
>> _______________________________________________
>> Tutor maillist - Tutor at python.org
>> To unsubscribe or change subscription options:
>> https://mail.python.org/mailman/listinfo/tutor
>
>
> ------------------------------
>
> Message: 4
> Date: Sat, 12 Dec 2015 12:23:32 -0500
> From: "Ken Hammer"<kfh777 at earthlink.net>
> To: tutor at python.org
> Subject: Re: [Tutor] Tutor Digest, Vol 142, Issue 10
> Message-ID: <E1a7ntK-0001EN-U8 at elasmtp-mealy.atl.sa.earthlink.net>
>
>
> I have a more simple-minded solution. The error appears to occur in testing without revealing the problem by use of $1 base price in the test. Using $10,000 as the base price in the test reveals absurd numbers. I think the real problem lies in the inclusion of base price inappropriately, twice as in:
>
>>dealerPrep = basePrice + 500
>>destinationCharge = basePrice + 250
>
> "basePrice" has no place in those two lines. 500 and 250 alone express the definition of the challenge.
>
> Ken
>
>
>
> In <mailman.13.1449939602.12987.tutor at python.org>, on 12/12/15
> at 12:00 PM, tutor-request at python.org said:
>
>
>
>>Send Tutor mailing list submissions to
>> tutor at python.org
>
>>To subscribe or unsubscribe via the World Wide Web, visit
>> https://mail.python.org/mailman/listinfo/tutor
>>or, via email, send a message with subject or body 'help' to
>> tutor-request at python.org
>
>>You can reach the person managing the list at
>> tutor-owner at python.org
>
>>When replying, please edit your Subject line so it is more specific than
>>"Re: Contents of Tutor digest..."
>
>
>>Today's Topics:
>
>> 1. Calculation error with a simple program (Jim Gallaher)
>> 2. Re: Calculation error with a simple program (Alan Gauld)
>> 3. Re: Calculation error with a simple program (Laura Creighton)
>
>
>>----------------------------------------------------------------------
>
>>Message: 1
>>Date: Sat, 12 Dec 2015 01:03:05 -0600
>>From: Jim Gallaher <jcgallaher78 at gmail.com>
>>To: tutor at python.org
>>Subject: [Tutor] Calculation error with a simple program
>>Message-ID: <566BC6A9.6090206 at gmail.com>
>>Content-Type: text/plain; charset=utf-8; format=flowed
>
>>Hi everyone. I'm reading through a beginners Python book and came up with
>>a super simple program. I'm not getting any errors and everything runs
>>through, but there's a logical calculation error. What the program does is
>>take an amount and calculate a couple percentages and add a couple fees.
>
>>For example, if I put in a value of 1, it will output 752.12 as the sub
>>total and 753.12 as the grand total. It's off by 1 on sub total and 2 on
>>grand total.
>
>>Thanks in advance! Jim Gallaher
>
>># Car Salesman Calculator
>
>># User enters the base price of the car and the program adds tax, license,
>>dealer prep, and destination charge.
>
>>print("Car Sales Calculator")
>>basePrice = int(input("Please enter in the price of the car: "))
>
>># Misc charges to be added to the total cost of the car
>>tax = basePrice * .07
>>license = basePrice * .05
>>dealerPrep = basePrice + 500
>>destinationCharge = basePrice + 250
>
>># Add the total misc charges together
>>subTotal = float(tax + license + dealerPrep + destinationCharge)
>
>># Add all the misic charges and include the base price
>>grandTotal = float(subTotal + basePrice)
>
>># Display the results
>>print("\nThe sub total is", subTotal)
>>print("\nYour grand Total is", grandTotal)
>
>>input("\nPress the enter key to close the program.")
>
>
>>------------------------------
>
>>Message: 2
>>Date: Sat, 12 Dec 2015 08:04:52 +0000
>>From: Alan Gauld <alan.gauld at btinternet.com>
>>To: tutor at python.org
>>Subject: Re: [Tutor] Calculation error with a simple program
>>Message-ID: <n4gkf4$e99$1 at ger.gmane.org>
>>Content-Type: text/plain; charset=utf-8
>
>>On 12/12/15 07:03, Jim Gallaher wrote:
>
>>> For example, if I put in a value of 1, it will output 752.12 as the sub
>>> total and 753.12 as the grand total. It's off by 1 on sub total and 2 on
>>> grand total.
>
>>Are you sure? Lets check the values...
>
>>> basePrice = int(input("Please enter in the price of the car: "))
>
>>=> 1
>
>>> tax = basePrice * .07
>
>>=> 0.07
>
>>> license = basePrice * .05
>
>>=> 0.05
>>> dealerPrep = basePrice + 500
>
>>=> 501
>
>>> destinationCharge = basePrice + 250
>
>>=> 251
>
>>> # Add the total misc charges together
>>> subTotal = float(tax + license + dealerPrep + destinationCharge)
>
>>=> 0.07 + 0.05 + 501 + 251 => 752.12
>
>>> # Add all the misic charges and include the base price
>>> grandTotal = float(subTotal + basePrice)
>
>>=> 752.12 + 1 => 753.12
>
>>Looks like Python got it right to me?
>
>
>
>
> --
> -----------------------------------------------------------
> K.F.Hammer Associates Ken Hammer
> management consultations Saint Johnsbury, VT 05819
> -----------------------------------------------------------
>
>
>
> ------------------------------
>
> Message: 5
> Date: Sun, 13 Dec 2015 15:44:33 +0800
> From: Crusier <crusier at gmail.com>
> To: tutor at python.org
> Subject: [Tutor] Beautiful Soup
> Message-ID:
> <CAC7HCj_dtvm6tb7S2RrM4ktPwUTbYv+wWHDJ7sdcQqsM8_8kww at mail.gmail.com>
> Content-Type: text/plain; charset=UTF-8
>
> Dear All,
>
> I am trying to scrap the following website, however, I have
> encountered some problems. As you can see, I am not really familiar
> with regex and I hope you can give me some pointers to how to solve
> this problem.
>
> I hope I can download all the transaction data into the database.
> However, I need to retrieve it first. The data which I hope to
> retrieve it is as follows:
>
> "
> 15:59:59 A 500 6.790 3,395
> 15:59:53 B 500 6.780 3,390................
>
> Thank you
>
> Below is my quote:
>
> from bs4 import BeautifulSoup
> import requests
> import re
>
> url = 'https://bochk.etnet.com.hk/content/bochkweb/eng/quote_transaction_daily_history.php?code=6881&time=F&timeFrom=090000&timeTo=160000&turnover=S&sessionId=44c99b61679e019666f0570db51ad932&volMin=0&turnoverMin=0'
>
> def turnover_detail(url):
> response = requests.get(url)
> html = response.content
> soup = BeautifulSoup(html,"html.parser")
> data = soup.find_all("script")
> for json in data:
> print(json)
>
> turnover_detail(url)
>
> Best Regards,
> Henry
>
>
> ------------------------------
>
> Message: 6
> Date: Sun, 13 Dec 2015 09:10:57 +0000
> From: Alan Gauld <alan.gauld at btinternet.com>
> To: tutor at python.org
> Subject: Re: [Tutor] Beautiful Soup
> Message-ID: <n4jcn1$qrm$1 at ger.gmane.org>
> Content-Type: text/plain; charset=utf-8
>
> On 13/12/15 07:44, Crusier wrote:
>> Dear All,
>>
>> I am trying to scrap the following website, however, I have
>> encountered some problems. As you can see, I am not really familiar
>> with regex and I hope you can give me some pointers to how to solve
>> this problem.
>
> I'm not sure why you mention regex because your script doesn't
> use regex. And for html that's a good thing.
>
>> I hope I can download all the transaction data into the database.
>> However, I need to retrieve it first. The data which I hope to
>> retrieve it is as follows:
>>
>> "
>> 15:59:59 A 500 6.790 3,395
>> 15:59:53 B 500 6.780 3,390................
>>
>
> Part of your problem is that the data is not in html format but
> is in fact part of the Javascript code on the page. And
> BeautifulSoup is not so good at parsing Javascript.
>
> The page code looks like
>
> <script type="text/javascript"
> src="../js/jquery.js?verID=20150826_153700"></script>
> <script type="text/javascript"
> src="../js/common_eng.js?verID=20150826_153700"></script>
> <script type="text/javascript"
> src="../js/corsrequest.js?verID=20150826_153700"></script>
> <script type="text/javascript"
> src="../js/wholedaytran.js?verID=20150826_153700"></script>
> <script type="text/javascript">
> var json_result =
> {"content":{"0":{"code":"6,881","timestamp":"15:59:59","order":"1175","transaction_type":"","bidask":"...{"code":"6,881","timestamp":"15:59:53","order":"1174","transaction_type":"",...{"code":"6,881","timestamp":"15:59:53","order":"1173",...
>
> followed by a bunch of function definitions and other stuff.
>
>
>> def turnover_detail(url):
>> response = requests.get(url)
>> html = response.content
>> soup = BeautifulSoup(html,"html.parser")
>> data = soup.find_all("script")
>> for json in data:
>> print(json)
>
> The name json here is misleading because it's not really
> json data at this point but javascript code. You will
> need to further filter the code lines down to the ones
> containing data and then convert them into pure json.
>
> You don't show us the output but I'm assuming it's
> the full javascript program? If so you need a second
> level of parsing to extract the data from that.
>
> It shouldn't be too difficult since the data you want
> all starts with the string {"code": apart from the
> first line which will need a little bit extra work.
> But I don't think you really need any regex to do
> this, regular string methods should suffice.
>
> I suggest you should write a helper function to do
> the data extraction and experiment in the interpreter
> using some cut n pasted sample data till you get it right!
>
> --
> Alan G
> Author of the Learn to Program web site
> http://www.alan-g.me.uk/
> http://www.amazon.com/author/alan_gauld
> Follow my photo-blog on Flickr at:
> http://www.flickr.com/photos/alangauldphotos
>
>
>
>
> ------------------------------
>
> Subject: Digest Footer
>
> _______________________________________________
> Tutor maillist - Tutor at python.org
> https://mail.python.org/mailman/listinfo/tutor
>
>
> ------------------------------
>
> End of Tutor Digest, Vol 142, Issue 11
> **************************************
More information about the Tutor
mailing list