django

All posts tagged django

replogo

Last time we looked at how to generate a very simple PDF using ReportLab and Django, ReportLab and Django – Part 1 – The Set Up and a Basic Example.  This time let’s make the PDF a little bit more interesting with some headers and footers.  ReportLab gives a pretty good amount of control when it comes to adding headers and footers to your PDF.  To start off let’s bring back the simple __init__ method we had from last time and add a header/footer method to it.

Now we have a method that will draw things onto every page of our header and footer.  Next we need to actually call that method.  Remember last time where we built the document using doc.build(elements) and ReportLab built a PDF using the flowable elements in the list?  Well this time we are just going to add some more parameters to let the build process know that it needs to also draw out the header and footer on every page.  Below is our print_users method from last time with the new parameters added to the build command shown in bold.

That’s it. Now our print_users method will print out a few lines on both the header and the footer of every page. You’ll notice there are two parameters, onFirstPage and onLaterPages. This allows you to have different headers and footers on the front page, which could be a title page, and on later pages that would require page numbers. Speaking of page numbers let’s get into that.

Page numbers on every page was a big challenge for me to find anything on. It was easy enough to put Page X on every page but what I really wanted was Page X of Y and that “of Y” portion was tricky. Luckily, I’ve figured it out and I will present it here for you now.

You can alter the way that the PDF is built using the canvasmaker parameter in the doc.build() process. These canvasmakers can help you inject new things on every page without using a header or footer. Below is the code that I found somewhere online to print the Page X of Y on every page down near the footer. I have placed this all within the printing.py file I created last time.

To use this we just add a new parameter to our doc.build() command. You can see it added below in bold.

There you have it. Now you have a great header and footer on every page, or every page after the first, and page numbers on all of them. How fantastic is that? Next time we will go a bit deeper into Tables and how to use them to line everything up really nicely. They will be usable wherever you want a flowable including your new headers and footers.

send-file-iconupload-files-icon-ts7tteez

As we continue to push forward on development our users have asked for a robust way of dragging and dropping images and linking them to a work order.  I was putting this off for a while because I thought it would be a pretty difficult task but after a few Google searches I found a great jQuery plugin from blueimp called jQuery-File-Upload.  Take a look at their demo and check out how awesome it is.  We didn’t need any of the fancy bells and whistles that came with the Basic Plus UI so this tutorial focuses on just the Basic but you can scale yours how you’d like.  The tutorial will also focus on getting it to work and save to a MySQL database through Django.

To start off let’s get the drag and drop area set up on our page.  To do that you’ll need to add these things to your HTML template, Django tags are added because that is what we are using to render the page.

image_upload.html

With the code above you should be ready to drag and drop files immediately, at least from the front end perspective. Now let’s set up the server side of things. Make sure you add your URL from the fileupload to your urls.py file. Then create a view for that URL that looks something like this.

views.py

The view handles all of the uploaded files one at a time and saves them to wherever we would like. In this case we are making a database entry that houses the file in an ImageField and we have a CharField for a name that we can change without altering the file path used in the ImageField. The JSON output is important as it will read the data sent back and can output our newly added images right to the screen.  That’s all there is to it.

Now there may be some circumstances where you would like to add more data to the request than just the images you have dragged over.  In our case we wanted the work order number to be attached as well and saved in the file name.  Using jQuery File Upload we can add data anytime something is dropped by adding a bind call to all fileuploadsubmit method calls.  I added the following lines of code in the HTML template just underneath the var url line.

This will add the workorder id to the POST request data on every call using the fileupload. Another thing to note is you can add a check to see if you really want to submit the uploads like I have done. If the workorder is not active do not upload any images. You can customize this any way you’d like since it is all just JavaScript and jQuery code.

Another problem I ran into was having multiple file upload elements on the same page. This was really frustrating and reading through their documentation they had a fix that worked pretty well with a couple of tweaks.

First, to make all of the drag and drop areas work independently of one another you change the fileupload elements from having an id of fileupload to a class of fileupload. This will need to change on the above code for bind as well. But you can’t simply change it on the $(‘#fileupload’).fileupload() becuase then it will upload once for every drag and drop element on the page. You instead have to take it a step back and add these lines.

This allows every fileupload classed element to take files individually. But this was annoying because users had to drop the files right on the button to make it work. As you can see there is a new parameter we’ve added called dropZone. This tells the fileupload what area of the page to watch for. If you want to make this area bigger or smaller that is totally up to you. I changed mine to dropZone: $(this).parents(".panel-body") to allow the user to drop images anywhere inside the panel body and they will start the upload.

I hope this was useful to someone. We are actually going in a different direction with our image uploads but we may need this in the future and I wanted to document how I got it all working with both jQuery File Upload and Django.

EDIT

Thanks to Tuan Nguyen for pointing out that in its default configuration each image is uploaded in its own request.  To join these into one request, open up the jQuery-File-Upload/js/jquery.fileupload.js file and search for the line singleUploadFiles: and set it to true.  You can also limit the number of files to be uploaded right below by changing limitMultiFileUploads:.

It’s been far too long since my last post.  A lot has happened and the holidays make things so busy but I’m back and ready to get back in to the swing of things.  To start off the new year I’m going to do a series of posts relating to ReportLab specifically with its application to Django.  I want to go over a few of the pitfalls I’ve experienced and how to get things set up.  ReportLab is notoriously tricky to get working and the documentation seems sparse to say the least.  I hope these posts help not only myself remeber how I did all of this but others who are struggling with the same problems.

ReportLab

The problem is simple, how to generate PDFs using Django.  In our on going rewrite of our system at work we are getting to PDF generation.  The old system uses wkhtmltopdf which is great if you just want something to render HTML out to a PDF and you don’t give much care to speed or formatting.  Wkhtmltopdf suffers from having to use HTML to format everything on the page.  This is limiting because it is difficult to add headers and footers, positioning objects on the page is tricky, and we have a bad problem with it printing blank pages due to random overflow of the elements.  We needed a better solution and we have found it with ReportLab.  ReportLab has some HTML support, and we’ll get to that in a later segment, but for the most part you are drawing elements, text, images, etc. directly onto a canvas that is then rendered as a PDF.  The speed increase for us was unbelievable.  The formatting and control that we get is unmatched.  Most importantly, the font support has given us cleaner paperwork than we ever thought possible.  I guess what I’m trying to say is ReportLab is great…once you get it to work.  ReportLab Documentation is broad but not specific enough for a lot of our concerns.  You’ll find yourself heading to Google for even the smallest of questions.  I hope to alleviate many of the common pitfalls with this series.

Getting Set Up

Start by installing reportlab into your virtual environment with pip install reportlab. The first thing to do is get a robust class set up for doing all of your printing.  I created a printing.py file that houses all of my basic ReportLab set up and functions.  Branching off from there I have my individual pages and paperwork that get combined and output to the user.  Let’s start with a simple printing.py file.

As you can see my class is very simple. I pass in a buffer to house my data that will be returned to the user, I’ll show this once we get set up, and a page size. ReportLab has a lot of built in page sizes but you can even set your own if you’d like by using your own width and height variables.

Simple Example – Printing Users

Now let’s add a simple method to print out all of the users of our application into a PDF and return it to the browser.  Inside of the MyPrint class I’m adding a method that will print the list of users.

The code is commented nicely but let’s walk through it.  We start by taking our buffer and putting it into what is a SimpleDocTemplate.  This is a ReportLab object that will generate a PDF with some standard values.  There are a ton of customizations you can make and even create your own document templates but for now let’s stick with the simple.  We set the margins for the page and the page size.  You can use inches or millimeters too if you’d like, ReportLab has a few built in measurement sizes that just need to be imported.

Next we have a list of elements.  This list will hold all of the ReportLab Flowables that will be generated on the PDF itself.  A Flowable is an object that has space and size on a PDF.  This can be a block of text, a table, an image, or you can even create a custom Flowable.  Whatever we put in this list will be generated in order on our PDF.

After that you’ll see some styles that we use.  These styles are going to be used for various elements like Paragraph styling.  After we get the style sheets I’ve added a new one that simply centers text and nothing more.

Now we come to the part where we are going to add elements to our document to be rendered.  I’ve used Django to grab my list of users and I loop through them one at a time to add their name to a Paragraph flowable.  A Paragraph flowable will take up the entire width of the area you give it.  In our case we have not confied the Paragraph to the inside of a table or to a column so each Paragraph will fill in the entire width of the document.

Once we have added all of our elements we simply build the doc.  This renders the PDF and places all PDF data into that buffer we gave the doc when we initiated it.  Close the buffer for good practice and return the contents of the PDF to the user.  Simple enough.  To run all of the code that I have shown I have a view in my views.py file that will run all of this and you can see how to access the printing.py class and method and output your PDF to your users.

That’s all there is to it. I hope this was enlightening on at least how to get started. Next time I will explain how to add headers and footers to your documents as well as a surprisingly tricky to find way of adding page numbers.

If you have anything you’d like to learn about for ReportLab and Django please leave a comment below.

ReportLab and Django – Part 2 – Headers and Footers with Page Numbers

Security-camera-sign

A couple of weeks ago it came out that there is a flaw in Django’s ImageField which could potentially allow for phising programs to be uploaded and grab cookies or do other malicious things.  While there will be no fix in Django directly you still need to take precautions on how you serve and receive files uploaded by your users.

Django has a page dedicated to fixing this exact issue.  Head over to their security guide and read up on the fixes.  They shouldn’t be too hard and shouldn’t affect any user experience.

Just wanted to post and let everyone know of the vulnerability.

Adding Custom Django 404 and 500 Pages

2_creative_404_pages_by_theollybanks-d3gmv9v

Django comes with a default 404 and 500 error pages which are dull and boring.  It would be much better if you add your own spin on those pages to inform your user of what happened.  Django makes this super easy by searching your project template files for a 404.html or 500.html file when it tries to render those pages.  This means all you need to do is make a nice Django template called 404.html and 500.html and place it in your project templates folder and you’re done!  You’ll have access to whatever context processors you are running for information, most of the time these pages won’t be user specific and will just display some error.

Please note that you won’t be able to see them unless you turn DEBUG = False in your settings.py file.  Once debug is off Django will render the 404 and 500 pages rather than the debug output.

Hope this helps people get their pages a little more user friendly and comfortable.

Django finally released version 1.6 of their Python based web-framework.  I use it at my job everyday and we ran into issues with the old system where they were using such an old version of Django that it was impossible to just simply update.  One of the goals with the new system we are creating is to stay as up to date as we possibly can with all of our technologies.  For us that is Bootstrap, jQuery, Django, and a few other third-party web plugins.  Here are a few of the notes I’m writing down for myself and for others to use if they run into issues in upgrading from 1.5.4 to 1.6.

Default session serialization switched to JSON

Django changed the way the session variable is being stored from pickle to JSON.  This is for the best as there is an exploit if the attacker knows your SECRET_KEY and has access to the CSRF tokens being sent by users and the session variable uses pickle.  Using JSON fixes this and Django has elected to make it the new default.  The issue is that some things, in our case timezones, are not JSON serializable.  If you’d like to still use pickle for your session variables just add SESSION_SERIALIZER = 'django.contrib.sessions.serializers.PickleSerializer' to your settings.py file.

Simplejson Deprecated

Simplejson has been deprecated and you should now use JSON.  This means you’ll need to change your imports from from django.utils import simplejson to just import json and you’ll need to change any code that uses simplejson to just json.

User Profile Deprecated

They have officially deprecated the use of user profiles attached to the default user in favor of custom user models.  Custom user models should be setup on initial project creation.  If you didn’t do this then you can use something like South to migrate your data.  The easiest way to create a custom user model is shown below by extending the base Django user and just adding to it.

settings.py

models.py

Now anywhere you would make a foreign key to User you’ll need to import settings and use settings.AUTH_USER_MODEL instead of User.

models.py

Those are all of the major things that I needed to change to update everything.  Granted they started to deprecate user profiles back in 1.5 we just hadn’t made the change yet and needed to now.  Hope this helps and if you find any other bugs or awkwardness when upgrading post in the comments below.  You can also read the Django 1.6 release notes on their site.

Django works wonderfully as a web framework but sufferes from sub-par data migration.  Luckily for all of us one of their developers has created an application called South that fixes this very problem.  South is meant to bring data migrations to Django and make it easy to make schema changes that are quickly applied to your database without much effort on your part.  Django does have a few built in features for dropping tables, creating tables, and will give you the code to enter to create the tables but it doesn’t help much when you are working on a production server that you don’t want to simply drop and readd tables all the time.

To get started just go through the installation tutorial.  Once you’ve done that you’re ready to migrate your data!  This is best done from the very creation of the project so South can help you from the beginning.  If you are starting a new project go here and follow along.  I’m going to focus a bit more on converting an already created project and making updates and changes to your projects.

Now that South is installed on your already created project you’ve been using for a while you’ll need to convert your app so South can interact with it.  In your console just enter the commands:

Done!  Now your app is ready to go with South.  South keeps track of the migrations using a few tables in your database which is why you need to syncdb before converting any of your apps.  After you have converted your apps you’ll notice you have some new .py files that will need to be uploaded to your repositories.  When South tries to do a migration it makes sure that you have all of the .py files that it has listed in its database for that app.  After the initial conversion, and you’ve uploaded the files, all other machines with the installation will need to run the command ./manage.py migrate myapp 0001 –fake to ensure everyone starts on the same page.  The –fake command at the end tells South to take note of what is happening but don’t actually do it because it’s already been done.

When you make changes to your models, from now on, you’ll need to use South to do the syncdb instead of Django.  Let’s say you add a few columns to a table, remove some others, rename even more, and add a bunch of new tables.  To have South take care of all of that just use the commands:

After running the first command South will try to document all of the changes, display them to the screen, and if all goes well it will tell you to run the second. When running the second command it will actually apply the changes to the database. That’s all there is to it. Occasionally, something will go wrong. If a problem occurs at the first command it’s most likely because you have a problem in your code that is making it fail to compile. If it fails in the second command there are more issues. You could have a partial migration which they will give you code to try and clean up but I’ve found it’s best to do it yourself as their solution is usually to drop a lot of tables. It’s kind of a headache when that happens and has turned off a few of us from South but for the most part we have had a lot of success with it. If you do make all of the changes by hand just be sure and tack on the –fake at the end of the second command to let South know that the changes were already made and that they shouldn’t try and do them again. Be sure and always commit your .py South files because even if I am in a different app trying to use South I will need all of the South .py files from all of the apps before the sync can happen.

Hope this helps a few people and explains a few things about the nuances of South. It really is a great tool and they are even trying to work it into the default installation of Django in the near future so that will be exciting. If you have any tips or tricks when using South post them in the comments below.