Big consequences of a good bug report

Last week I had fixed a trivial bug (a leftover from a former change where a function’s return was changed, but one place of its usage managed to escape to be not converted to properly treat the changed return). It seems to simultaneously have fixed a number of other bugs (the discussion may be found in the bug tracker issue). The little (a few characters) bug turned out to create both performance issues, and clipping of characters, so it had big impact on LibreOffice on Windows (with DirectWrite, e.g. when OpenGL is used).

The problem became trivial both to find and fix, because of great bug report by Telesto, who not only filed the report, but also had provided every relevant piece of information, including terminal output accompanied the problem manifestation. I cannot emphasize enough the importance of this: the effort of the bug reporter makes a difference. Without the effort, some problems remain very difficult for developers to be tracked down and get fixed.

I write this to praise Telesto‘s great job, and urge every reporter of a bug to follow this great lead.

Advertisements

Windows Unicode API usage in LibreOffice

Windows still provides two sets of many of its Win32 API functions taking or returning strings: a legacy “Ansi” (functions named like fooA) and Unicode (named like fooW; available since Windows NT, and in Windows 95 with Layer for Unicode – and thus on any Windows OS supported by LibreOffice).

The “Ansi” functions take 8-bit strings in current codepage (single- or multibyte). The repertoire of characters representable in those strings is, naturally, limited to that codepage (that is either setup in system’s Language for non-Unicode programs, or explicitly set by running application). Unfortunately, unlike in other contemporary OSes, Windows doesn’t allow setting its locale to use UTF-8. If a string arrives to such a function that contains characters outside of that set, the string content will be altered, and functions’ behaviour might change unexpectedly.

“W” versions of those functions take UCS-2 strings, that are able to represent most of Unicode range (I am unsure if those strings are actually UTF-16, and so are able to represent the full Unicode repertoire, but anyway, even UCS-2 is much wider than most of single- or multi-byte codepages).

In last two weeks, we have replaced many places (0, 1, 2, 3, 4, 5, 6, 7, 8, 9, A, B, C, D) in LibreOffice codebase where legacy “A”-functions were still used, with explicit calls of their “W”-counterparts, removing redundant conversions of strings from LibreOffice internal UTF-16 string representation to 8-bit strings and back. One of most significant effects might be on file-management functions, where such conversions could alter paths/names containing Unicode characters not representable in currently selected 8-bit codepage, and lead to failed file operations. One example of such problems is tdf#103525.

The changes are included into master towards 6.0.

App compatibility manifest

Last week, we’ve added application compatibility manifests to LibreOffice libraries and executables. This is intended to allow the program to run in current OS context, instead of switching to a previous (namely, Windows Vista in case of absent manifest section) OS compatibility context. The effects of this are listed in a MS KB (this is alternative link just in case).

Currently we declare compatibility with Windows 7 through Windows 10. This change is integrated into current master towards LibreOffice 6.0.

LibreOffice MUFFIN

Recently, TDF has published its new MUFFIN concept. This publication has provoked a mixed response, with many negative reactions.

When we deal with UI, any possible change brings controversy, and it’s always very aggressive when someone finds that a function they are used to use some way had changed its position. OTOH, there is always a substantial part of community that aggressively promotes “modernization” of current UI.

We all remember how different projects made their UI changes. We remember cases when programs with huge user base made radical changes without an option to keep old UI for those who prefer it. And it forces many of us to resist any sign of possible UI changes in software we use.

Many believe that “design” and “marketing” are swearwords that are used when a company tries to make “purchasers” believe they need something they actually don’t. So, any use of these words in a press release makes yet another chunk of user base to get angry.

There are people that don’t care of UI changes, but have strong opinions that there are much higher-priority tasks to do, like improving stability, compatibility and so on. And when they hear of an effort in an area they regard as unimportant, they get upset. Some of them even have donated to TDF with hopes that donations would go to those goals, and now “see” that TDF wastes their money to pay some designers/marketologists instead of doing Right Things (TM).

Many see the news about UI and immediately realize that there’s indeed something new in UI that is announced. They don’t care to read (let alone comprehend), and so skip any text for screenshots/mockups, and get their views on the matter based on their perception of the graphics. And many of them are angry because they see something that disappoints their expectations of e.g. “ribbon-like” interface. Or they suppose that TDF fools them trying to make believe that there’s something new in the UI options (besides NotebookBar) when actually all that is presented is already possible for long time. “Ah, they try to show to you something old, and pretend that it recently have made a great work to create something new! Smart move!”

Some think that new UI elements are not ready for use and need to be polished in a different branch before inclusion into a LO release.

Many believe that UI changes must begin with UI implementation being rebased to some other library like QT, and that any effort that doesn’t do this is waste of time.

People seem to misunderstand what MUFFIN is.

Thankfully, there are some people in LO community who feel interested in design issues. They form our Design team. Those people are not necessarily developers, and if they are, they have their own interests and priorities, independent of TDF’s or some specific user’s. That’s a great feature of open source community, where each one is able to find an area of interests to apply their effort. And it doesn’t matter if you want LO to get a bug-fix/compatibility feature, and it doesn’t matter if I do some work on those fixes/features: those other people don’t need and aren’t forced to wait for me or help me; they volunteer to do their own contribution in their area in their spare time, and announce their results when they feel appropriate. So anyone who argues that some other things should have been done instead of this work, is misguided. If we would prevent the Design team from doing this, we wouldn’t get more man-power dedicated to your preferred tasks; rather, we would just had lost the man-power (and more important, we would loose those contributors, because there wouldn’t be any reason for them to stay with the project).

The Design team clearly sees the challenges that LO faces with regards to UI changes. On one hand, many want some “refresh”; on the other, many deny the very possibility of it and are afraid of it. There are issues in UI not following different OSes/DMs conventions (making LO to feel not native to these OSes/DMs). And with MUFFIN, the team had IMO made a very smart move.

MUFFIN isn’t a new interface itself. It is even not an interface concept in a usual sense of the term! What it actually is is a public statement, a promise made by LO to its users; and this statement and promise is that: LibreOffice believes that it should be pleasant and easy to use by anyone; and it will never take steps (like other softwares sometimes do) that will sacrifice one group of users’ preferences just to please another – instead, it promises to make its UI as much configurable and modular as it required to allow you, me and anyone to build their own UI of choice. The message is that LO will e.g. continue to provide toolbars: this is not a “first step after which some evil designer will inevitably drop support for good old UI”! And in the same time, LO will not ignore those numerous current and potential users that want another interface: their interests aren’t of lower importance.

If someone still believes in a conspiracy theory that will eventually deprive you from your dear toolbars, remember that LO is being developed by those developers who prefer toolbars themselves, and there’s no ultimate authority to make decision to drop it regardless of those developers’ PoV.

So, MUFFIN is not a NotebookBar or a Single Toolbar. It is a message to users, and a guideline to developers who make UI-related changes. MUFFIN is not the four presented UI options; rather, they are just demonstration of commitment to make it easy to any group to customize.

The four presented UI options are not meant to be necessarily something new. Of course, you can use standard toolbars, or customize them to let only one custom toolbar visible, or use sidebar with any combination of toolbars: the UI components and their combinations aren’t new. The UI components aren’t perfect, and the concept doesn’t claim they are. The concept is orthogonal to any framework used to implement UI components. So, those who are not pleased because the UI isn’t new or isn’t perfect aren’t right when they blame MUFFIN for that. MUFFIN is a promise – so it’s not something to be kept in a separate development branch; and it’s not something that needs screenshots. I’d say that providing screenshots in the press release actually distracted and misguided many readers from the real message.

TDF didn’t spend money to make this happen. It used a research that is publicly available. It doesn’t spend your money to pay some greedy designers: it uses its community’s great power. So no need to get angry that your money are wasted. (As a side note, I’d suggest people to get familiar with TDF role and understand that it doesn’t develop LO itself; instead, it provides home and shelter to community, and promotes the software. Understanding this can help you avoid disappointment when you find out that your donations go to other needs that you imagined.)

I believe that MUFFIN is a great message if you make an effort to hear it. Please do. And if you believe that there’s something to improve, please contribute and become part of community!

My first hack-(rest-of-the)-week at Collabora

I must say that I’ve been attentive reader of TDF feeds almost from the very beginning of its history, when I couldn’t think to develop for LibreOffice. And reading posts of developers, where they described what they had done at their hack-weeks at SUSE, made me really envy them…

So here is my first hack-(rest-of-the)-week at Collabora Productivity! That’s cool!

And when it happened, it turned out that I need to finish a work that I started this week. What a pity! Well, the task itself was really interesting, and I’m glad that it is in a close-to-be-finished form in gerrit… but still. So, only hack-weekend 🙂

As I didn’t have that much time, I couldn’t afford doing something big and lengthy. So I turned to my favorite topic: SAXParseException.

Here I need to tell you what is it. Until LibreOffice 5.0.4, many errors in XML that LibreOffice reads were simply silently ignored. Note that both LO’s native format, ODF, as well as its most popular competitor, OOXML, are XML formats. It means that the file could be read incorrectly, some data could be dropped, some bugs could go unnoticed, but file opened as if everything was OK. Data loss could go unnoticed.

In January 2015, two days before my daughter’s 3rd birthday , a named hero Michael Stahl took required steps and committed a patch that put that madness to end. Now every error in the XML data that LO reads is reported to user as famous SAXParseException. This annoys users, but it yells loudly “Fix me!”, and it has already enabled us to find and fix quite a number of errors that previously went unnoticed.

So, I must say that I like to hack on these bugs. But since I started to work full-time as developer, I had not that much time for them. And now I took bug 99227 for the rest of my hack-week.

It turned out to be LibreOffice OOXML export filter fault, that caused loss of drawings in footnotes/endnotes. It also created an invalid XML, which was caught on following import, telling that it found “Extra content at the end of the document”.

I’m happy to say that the patch is on its way (it’s on review, and after some rounds of fixing my evident mistakes pointed out by merciless reviewers, it will be pushed).

It may seem odd that I feel so enthusiastic about changing hacking on LO by … hacking on LO. 🙂 But actually, my daily work is so great, I really love it! and being able to choose tasks myself is the only part that I miss sometimes… and that I get now.

Thanks to Collabora Productivity, to all fellow hackers, and thank you for reading this 🙂

Integrating LibreOffice with SharePoint – part 2

In my previous post, I discuss LibreOffice‘s CMIS integration with SharePoint. This post, as promised, is dedicated to WebDAV method of accessing SharePoint servers from LibreOffice.

Basic WebDAV support is already present in LibreOffice, which inherits it from OpenOffice.org. You may put SharePoint URL into OS’s File Open dialog, and browse and open documents as if they were on local file system. But to claim SharePoint WebDAV support, LibreOffice should provide ease of use no less than its competitors.

Existing users of MS Office are used to access the documents on SharePoint server using web browser integration. This allows to navigate to a document and open it in associated office program simply clicking on its link on web page, or using the page’s options to check-out document, open it for reading, etc. A document that is open this way isn’t downloaded by browser to be opened by application locally, but is open by office application via WebDAV, thus presenting easy way to work with document, including saving your work to proper place automatically. This integration is now available for all modern browsers (the level of feature support vary, though).

sp-webdav1

There are two mechanisms used to enable such integration. The first (and older) one is using ActiveX control/browser plugin. This method works for IE (classic mode) using OpenDocuments ActiveX control (and also used to work for Firefox and Chrome using NPAPI plugin, but since NPAPI was deprecated, it doesn’t work there anymore). In essence, a JavaScript click handler on the page instantiates the control, and then uses its methods to open the document with corresponding application in required mode. With all implied security considerations.

The second (newer) method is using custom URI schemes. This requires that corresponding URI handler be registered in OS/browser, and the handler application be able to recognize these URIs and extract actual WebDAV URIs required to access the document on server from them.

Both methods were lately addressed in LibreOffice development by Collabora Productivity. To implement first method, a custom ActiveX control was coded, and is now awaiting review on gerrit.

To utilize SharePoint custom URIs, LibreOffice was taught to recognize MS Office’s existing custom URIs. This is required to allow using LibreOffice to open MS Office’s documents from SharePoint on those computers that don’t have MS Office installed. But that’s not enough; we should be able to treat ODF independently, e.g. to allow for configurations where both LibreOffice and MS Office co-exist on a workstation, and must handle their respective native formats. To achieve that, we introduced our own custom URI scheme, vnd.libreoffice.command. Now LibreOffice can accept any of these URIs, and successfully open the document in proper mode.

But for LibreOffice to open them, these URIs must be passed to it from the SharePoint web page. We added registration of our own custom URI handler into installer for Windows OS, and also we now register LibreOffice as MS Office URIs handler if user chooses to register MS formats to open with LibreOffice.

If you choose to use the second (custom URI) integration method, and you don’t need MS Office as handler for DOCXs, PPTXs etc., then you don’t need to configure your SharePoint server for that to work. All that you need is to install recent enough LibreOffice (v.5.3+), and configure it at installation time to open MS Office files.

But if you choose either first method (ActiveX), or second method with coexisting LibreOffice/MS Office workstations, then you will need to configure SharePoint server for that. In both cases, you will need to edit DOCICON.xml file, that  may be found in server’s %ProgramFiles%\Common Files\Microsoft Shared\Web Server Extensions\XX\TEMPLATE\XML. This is an XML file, and we should edit sub-elements of ByExtension element. Such sub-element has the following syntax:

<Mapping Key="odt" Value="icodt.png" EditText="LibreOffice" OpenControl="LOSPSupport.OpenDocuments" OpenApp="vnd.libreoffice.command"/>
  • Key = file extension
  • Value = icon file
  • EditText = Text on “Edit with application” UI button – required for edit option. Optional.
  • OpenControl = ActiveX OpenDocuments control name (currently IE 32-bit only). Optional.
  • OpenApp = custom URI scheme to handle this. Independent from and appears to have priority over OpenControl. Available in 2013 SP1. Optional.

You should use either OpenControl or OpenApp, depending on which integration method you have chosen. Do it for those extensions that you want to be handled by LibreOffice. Of course, you should backup the file prior to modifying it.

After you have made all necessary modifications, to activate the settings, you need to restart IIS. To do that, you may execute command iisreset from the server’s administrator command line.

That’s it, if you did everything right, LibreOffice should now seamlessly integrate with SharePoint website, and be automatically launched when you choose to open files from there, without downloading the file to local machine first.

This is the second of two posts that discuss the current state of SharePoint integration support in LibreOffice. Thank you for reading, and enjoy the best free office suite ever!